Governed patch deployment from policy to completion

Prerequisites

• Role with patches.view permission for read-only access
• Role with patches.manage permission for creating policies, ring sets, deployments, and approving/rejecting patches
• Role with patches.deploy permission for skipping individual deployment hosts
• Role with hosts.view / hosts.manage for service group management
• At least one organization with managed hosts that have agents installed and reporting heartbeats
• Hosts must be sending pending patch data via host-info collection (automatic with agent v1.7.0+)

Creating a patch policy

Patch policies control auto-approval rules, safety thresholds, and pre-flight behavior. Policies are org-scoped -- one active policy per organization.

1. Navigate to Patch Management > Policies
2. Click Create Policy
3. Select the target organization
4. Configure the fields below
5. Click Save

Policy fields

Policy overrides

Overrides let you customize policy settings for specific locations or host groups without creating entirely separate policies. Overrides use sparse merge -- only fields you specify override the base policy. All other fields inherit from the base.

1. Open the policy detail page
2. Click Add Override
3. Select the target: a location or host group
4. Set only the fields you want to override (leave others blank to inherit)
5. Set the priority number -- higher priority wins when multiple overrides apply to the same host
6. Click Save

Creating maintenance windows

Maintenance windows define when patching and reboots are permitted. Hosts configured with the "Maintenance Window" install schedule will only begin patching inside an active window.

1. Navigate to Maintenance Windows
2. Click Create Window
3. Enter a name and select the organization
4. Choose the schedule type
5. Set the start and end times (24-hour format)
6. Set the timezone (e.g., America/New_York) -- all MW times are evaluated in this timezone
7. Add targets: select host groups or individual hosts
8. Click Preview Schedule to verify the next 3 occurrences
9. Click Save

Schedule types

Creating ring sets and rings

Ring sets define the phased rollout structure for patch deployments. Each ring set contains ordered rings that deploy sequentially.

1. Navigate to Patch Management > Ring Sets
2. Click Create Ring Set
3. Enter a name (e.g., "Linux Servers" or "Windows Workstations")
4. Toggle Auto-deploy if approved patches should automatically create deployments
5. Optionally set a classification filter to limit to security-only, critical-only, etc.
6. Optionally configure script hooks: pre-script, post-script, synthetic test script
7. Click Save

Adding rings

Add rings in deployment order (ring 0 deploys first). A typical three-ring setup:

For each ring, add members: select host groups, individual hosts, or service groups.

Ring configuration

Install schedule modes

Each ring has an install schedule that controls when patching begins for hosts in that ring.

Reboot policy

Each ring has its own reboot policy controlling what happens after patches install:

Canary settings

Service groups

Service groups enable coordinated patching of dependent servers in multi-tier applications (e.g., Web → App → DB). Tiers patch in dependency order to avoid breaking service dependencies.

1. Navigate to Service Groups in the Policy sidebar section
2. Click Create Service Group
3. Enter a name (e.g., "Production ERP Stack") and select the organization
4. Add tiers in dependency order (tier 0 patches first)

Tier configuration

Each tier references a host group -- the hosts in that group become the tier's members.

Example tier setup

Add the service group as a ring member (same as adding a host group). During deployment:

• Service group hosts are never canary -- tier ordering IS the validation mechanism
• Tier 0 completes first, then tier 1, etc.
• If a tier fails its success gate, the deployment pauses
• The Max Concurrent setting controls how many hosts patch simultaneously within a tier

Managing available patches

As agents report pending patches, they appear in Patch Management > Available Patches. Patches matching auto-approve rules are approved automatically (shown as "System" in the approved-by column). Others stay in "Pending" status for manual review.

Manual actions

Creating a deployment

If auto-deploy is enabled on a ring set, deployments create automatically when patches are approved. For manual deployments:

1. Navigate to Patch Management > Deployments
2. Click Create Deployment
3. Select the ring set
4. Choose patches to include
5. Configure overrides if needed (max host retries, max duration, pre-download, script hooks)
6. Click Save -- deployment starts in Draft status
7. Click Start Deployment

Deployment options

Deployment lifecycle

When you start a deployment, the engine processes it through the following phases:

1. Ring 0 expansion: The engine expands ring 0 members into individual host rows, skipping empty rings automatically
2. Canary selection: The first N hosts are marked as canaries
3. Canary patching: Canary hosts begin the per-host pipeline immediately
4. Canary wait: After all canaries complete, the system waits for the configured canary wait period
5. Remaining ring hosts: Non-canary hosts begin patching (respecting install schedule mode)
6. Success gate check: When all ring hosts finish, the success rate is evaluated against the configured success gate percentage
7. Cooloff: The system waits for the configured cooloff period before advancing
8. Variance detection: For ring N > 0, if patches exist that were not tested in any previous ring and "Block Untested Patches" is enabled, deployment enters "Variance Approval Required" status
9. Next ring: Steps 1-8 repeat for each subsequent ring
10. Completion: When all rings finish, deployment moves to "Completed" (or "Completed with Failures" if any hosts failed/timed out/were skipped)

Per-host pipeline

Each host goes through this sequence (phases are driven by job completion callbacks):

Monitoring deployment progress

The deployment detail page shows real-time status across all rings and hosts.

Deployment statuses

Host statuses to watch for

Variance detection

If a higher ring has patches that were not tested in any previous ring and the policy has "Block Untested Patches" enabled, the deployment enters "Variance Approval Required" status. Click Approve Variance to continue, or disable "Block Untested Patches" in the policy.

Deployment actions

Auto-retry on failure

When a host fails during any pipeline phase, the system automatically retries up to the configured Max Host Retries (default: 1).

• Retry resets the host to "Pending" and restarts the entire pipeline from scratch (download, preflight, install, etc.)
• All per-run state (fingerprints, jobs, error messages) is cleared
• Retries are tracked and visible on the host detail
• Auto-retry does NOT apply to rollback failures (prevents infinite loops)
• The circuit breaker still fires for install-phase failures even during retries

Circuit breaker

The circuit breaker protects against widespread failures from a bad patch. When a patch KB fails on 3+ hosts across the entire account (default threshold), the system auto-rejects that KB in all organizations.

• Threshold is configurable per policy via the Global Failure Threshold setting
• Counts failed and rolled-back hosts for the same KB across all account organizations
• Once triggered, the KB is rejected and no further deployment attempts are made
• To re-deploy after fixing the root cause, manually re-approve the KB

Deployment auto-cancellation

Two automatic cancellation mechanisms protect against stuck deployments:

MW auto-skip

Hosts using the "Maintenance Window" install schedule that have been in "Pending" status for over 48 hours with no maintenance window configured are automatically skipped. This prevents misconfigured hosts from blocking an entire ring indefinitely.

Post-deployment

Completion summary

When all rings complete, the deployment builds a completion summary including per-ring success rates, failed host details, and total patches installed. The deployment status is:

• "Completed" -- all hosts succeeded
• "Completed with Failures" -- some hosts failed, timed out, or were skipped

Reboot handling

Reboot behavior is per-ring. After patches install, the reboot policy determines when the host reboots. If set to "Maintenance Window", the reboot is scheduled for the next MW. If set to "Manual", the operator is responsible for rebooting.

Deployment lifecycle flow

Permissions reference

Troubleshooting