Sheet REL-13 · Endpoint Management Manual
Privileged Access Management
Vaults, credential checkout and approval, step-up reveal, break-glass and scheduled rotation.
Scope
How to manage vaults and credentials in the PAM console, check credentials out, reveal secrets, use break-glass and schedule rotation. Credentials are stored in the Portal credential vault.
Relay’s PAM console manages privileged credentials stored in the Portal credential vault — Relay never stores a secret at rest. This guide covers vaults, credentials, checkout/approval, reveal, break-glass, and the compliance views.
Related documents:
Availability: Portal is a hard dependency for privileged access. When the vault cannot be used, the PAM page says why, and the three reasons need different people:
- “Relay has no usable Portal service client for this account’s Portal tenant… Portal itself was not contacted.” (
portal_integration_not_provisioned) — Portal is fine; this tenant was never provisioned, or its credential was revoked. An administrator fixes it (SaaS Portal manual, “Portal integration states”); retrying cannot help.- “Portal did not answer…” (
portal_unreachable) — Portal is down or timing out. Retry after the time shown.- “Portal refused Relay’s service credentials…” or “…lacks the scope…” (
portal_credentials_rejected,portal_scope_refused) — an administrator rotates the service client or grants the named scope.Nothing fails silently: the PAM page shows the failure with its remediation and reference, and disables credential/vault creation, and the Sessions, Approvals and vault-credential lists each show their own failure rather than an empty list — an unusable vault never looks like “nothing is checked out” or “no one is waiting for approval”. A refusal by the vault’s own policy (you lack the vault permission, or an approval or step-up is needed) is a different message: “Portal’s vault refused this action for you”.
Vaults
Every credential lives in exactly one vault. Each tenant has an undeletable Default vault; imports always land there.
Create a vault (pam_vaults.manage)
Console: PAM → Vaults → New Vault, or
Create one vault per ownership domain (team, service, customer). The creator principal becomes the first owner — then set real owners deliberately.
Owners and permissions (pam_access.manage)
A vault carries two ACL surfaces, edited from the vault’s detail panel:
every capability on the vault’s credentials plus vault management (rename, ACL edits, moves, deletes).
- Permission rows: per-principal grants of
can_checkout,can_reveal,can_rotate,can_manage.
How access is evaluated for a credential action:
- Vault owner → allowed.
- Permission rows exist → a matching row must grant the action.
-
No owners and no rows → open to anyone with the product scope. This is the out-of-box Default-vault posture — convenient for evaluation, wrong for production. Set owners deliberately.
-
Owners but no rows → owners only (adding the first owner locks the vault).
Picking a principal. Users and groups are chosen by name from the Portal directory synced into this account — search for the person or group and select them; you never type an id. Service clients have no directory to browse, so directory has not synced yet the picker says so rather than presenting an empty list.
Principal names show as resolved display names; a grayed “unresolved” entry means the principal no longer exists — clean it up.
Moving and deleting
-
Move a credential: vault detail → credential list → Move (requires manage on both the source and destination vault).
-
Delete a vault: refused while it still holds credentials (move or delete them first) and always refused for the Default vault.
Credentials (identities)
Create (pam_vaults.manage)
Console: PAM → New Identity. Fields:
name— e.g.svc-sql-prodtarget_ref— opaque reference to the rotation target (e.g.endpoint:9931)-
policy: rotation interval, default checkout TTL, approval required, break-glass allowed
-
step-up mode —
service_attestedorportal_assertion(see Reveal below). This is a field of the credential, not of its policy block. -
the initial secret (sent once to Portal, never shown again)
- vault (defaults to Default)
Bulk import
PAM → Import loads many credentials at once; all rows land in the Default vault — move them into their target vaults afterwards.
Metadata
Lists and detail views show metadata only (vault, policy, provider). No list or detail response ever contains a secret.
Checkout and approval
Checkout (pam_vaults.view + vault ACL)
Checkout on a credential asks for a reason and duration and returns a time-limited session token. The token is shown once and is required for reveal; it expires automatically or when you check in.
When approval is required
If the credential’s policy requires approval, checkout returns pending instead of a session:
-
Your request appears in PAM → Approvals for approvers (
pam_vaults.manage). You cannot approve your own request — Portal rejects self-approval. -
Once approved, re-run the checkout (the console’s pending dialog offers Complete checkout) — the approval is single-use: it grants exactly one session, then shows as consumed.
-
A denied request ends the flow; check the approver’s reason.
Your active checkouts
Checkouts you make appear in a strip at the top of the PAM page with a live countdown to expiry. Because Portal issues a session token once and Relay stores no session state, that strip is browser-local: it lists the checkouts made in this browser. Losing it is harmless — the session still expires server-side, and PAM → Sessions shows every active checkout in the tenant.
When the checked-out credential targets an endpoint (its target_ref is
endpoint:<id>), the strip also offers Launch SSH and Launch Desktop,
which open a remote session on that endpoint with the checkout already
shortcut. You can also reach the same place from the endpoint: open Terminal
or Remote Desktop, choose PAM Checkout, and pick the session.
Check in
Check In ends the session immediately. Expired or checked-in sessions are rejected at use time — a stale token cannot reveal anything.
Reveal (step-up enforced)
Revealing plaintext requires an active checkout session plus the credential’s step-up policy:
-
Service-attested (default): Relay enforces its own operator step-up — the
pam_vaults.managepermission on your authenticated session — and attests you to Portal (relay:user:{id}+ your real authentication method; MFA-verified sessions attest asmfa). The attestation is audited verbatim in Portal. -
Portal assertion: the credential demands a Portal mobile-action challenge. Complete the challenge in the Portal app; the console forwards the assertion. Without it Portal refuses the reveal.
The console flow is: Reveal → (checkout with reason) → secret shown once, auto-cleared after 30 seconds → session checked in when you close the dialog. Every reveal is audited on both sides (Relay audit row + Portal audit event id).
Break-glass (pam_vaults.break_glass)
Emergency access when the approval chain is unavailable:
- Requires a justification of at least 20 characters; rate-limited.
-
Bypasses approval but NOT policy: credentials with break-glass disallowed refuse it.
-
Fires a critical notification to account admins and a heightened Portal audit event (
pam.break_glass.used). -
Returns the secret directly — treat the credential as exposed and rotate it as soon as the incident allows.
Oversight
-
Active checkouts (PAM → Sessions): tenant-wide list of live sessions — who has what checked out, since when, until when.
-
Session history: per-credential audit trail (identity detail → Session History).
-
Approvals queue: pending/decided requests with reasons.
-
Compliance report (top of the PAM page): Portal’s rotation/session KPIs for the tenant.
-
Relay audit log: every console mutation and secret-out action also writes a Relay audit row (actor, action, Portal resource id, and for reveals the Portal audit event id — never the secret).
Scheduled rotation
Portal decides when a credential should rotate (its policy interval); Relay’s agents do the rotation, because only they can reach the target inside the customer network. Configure the execution side per credential:
-
Provider:
os_account(local/domain account on the linked endpoints), endpoint, script). Sensitive fields are encrypted. -
Linked endpoints: rotation dispatches one agent job per linked endpoint. A credential with no linked endpoints cannot rotate — the run fails with “no agent jobs could be dispatched” rather than pretending to succeed.
From a Portal credential to a rotating schedule
Three things must be true before a credential rotates. Doing them in this order avoids the two mistakes that produce a schedule which looks healthy and never rotates anything.
1. Link the credential to the endpoints it lives on. Rotation targets come from the identity→endpoint association, not from Portal — Portal’s identity model carries no OS account name, so the local account to change is Relay’s data. Skip this and the schedule is created but its first run fails its precheck.
username is the local account on that endpoint — the account the agent
becomes, and the one rotation changes. The same link is what the terminal and
script run-as pickers offer, so creating it makes the credential usable for
interactive work as well as rotation. GET the same path to see what is
offering it there (the Portal credential itself is untouched).
flight or quarantined**. That is not about the running job — it will settle correctly either way — it is about the machine. A rotation in flight is putting a new secret on that endpoint, and a quarantined one already put one there without confirming the result. The link row is Relay’s only record of where that secret went, so removing it strands the machine with a credential nothing tracks: future rotations skip it, the copies drift apart, and when the account breaks there nothing points at why. Wait for the rotation to settle, or resolve the quarantine first.
Repeat per endpoint: one credential can target many machines, and rotation dispatches one agent job per link.
Linking is refused if Portal does not have that credential, so a typo’d id fails here rather than later as a picker entry that errors on use. Re-linking local account — remove the link and re-create it to change the account.
If you are enrolling a new local account rather than pointing an existing credential at a machine, use the endpoint’s Enroll in PAM action instead: it generates a password, creates the credential in Portal, resets the account on the machine, and creates this link for you in one step.
Portal identity id, the local username to rotate, the credential_type, the
interval, and the provider settings:
Leave rotation_provider out to let it resolve from credential_type
manual). The response tells you whether the schedule can actually run:
target_endpoint_count is how many endpoints it will dispatch to, and
warnings names anything that would stop it — no linked endpoints, or the
schedule being inactive. An empty warnings array is the confirmation that
step 1 was done correctly.
at rest keyed to the schedule’s own row and is never read back by any
route — responses carry only has_provider_config: true. To change it, send a
new one; there is no way to retrieve the old.
3. Wait, or start it now. The schedule becomes due at next_rotation_at,
which defaults to now + the interval. `POST
immediately. That arms the schedule rather than rotating inline, so an
operator-triggered rotation takes the identical fenced and verified path as a
scheduled one.
state, and changes it. cannot repoint a schedule at a different Portal identity — that is the row’s identity, not a setting, and repointing it mid-flight would rotate the wrong account. Create a new schedule instead.
in two situations, and the message says which:
-
A rotation is in flight (
pending). A Portal intent is fencing it and the agent callback is still expected. Wait for it to settle. -
The credential is quarantined (
unresolved). Relay is holding the applied secret because it may be the only copy of what the account now requires. Deleting the schedule would destroy that silently — resolve it at the secret deliberately and records why.
Renaming stays available while a rotation is in flight; only the provider is frozen, because a change there would land on the running attempt’s callback.
To stop rotating a credential without losing its configuration, set
is_active: false — the loop skips inactive schedules. Delete removes it
entirely.
Rotation states you will see: pending (jobs dispatched), confirmed
(every job set and verified the new secret, and Portal committed the new
version), failed with a specific reason, awaiting manual for
manual-provider credentials (a human must act), and failed after retries
once three attempts are exhausted — that one raises a high-severity alert.
A rotation is never half-done. Either every target took the new secret and Portal committed it, or Portal keeps the old secret and the whole rotation restarts from the beginning.
What is not in the PAM console
Machine credentials that Relay’s own subsystems use to operate — SNMP communities and v3 users, discovery scan logins, backup platform and storage credentials, infrastructure-bridge and LLM-bridge connector keys — are not vault credentials. They live in Relay, encrypted, and are managed from their own subsystem pages (SNMP, Discovery, Backup, Integrations), under those subsystems’ permissions.
The dividing line is who uses the credential. A person checking out privileged access → Portal vault, with approvals and audit. A background poll running every minute unattended → a Relay service credential. Do not put an administrator account in a service credential to avoid the approval flow.
Permissions quick reference
| To do this | You need |
|---|---|
| Browse credentials/vaults/sessions/approvals, checkout, checkin | pam_vaults.view (+ vault ACL) |
| Create/edit/delete/import credentials, create/rename vaults, reveal, decide approvals | pam_vaults.manage (+ vault ACL) |
| View rotation schedules and quarantined credentials | pam_vaults.view |
| Create/edit/delete a rotation schedule, rotate-now | pam_vaults.manage |
| See which credentials are pointed at an endpoint | endpoints.view + pam_vaults.view |
| Point a credential at an endpoint, or remove it | endpoints.manage + pam_vaults.manage |
| Break-glass, and revealing a quarantined rotation secret | pam_vaults.break_glass |
| Vault owners/permissions, credential move, vault delete | pam_access.manage (+ vault manage in Portal) |
Relay’s permission gates are defense-in-depth: Portal’s vault ACL is the final authority, and a Portal refusal shows as a 403 with Portal’s error code.