Push a release the fleet cannot half-take shipped

A fleet-wide change should end in one of two states: every node that matters runs it, or nobody does. This guide marks the nodes that matter, starts a canary rollout with a release deadline, and watches the release gate decide. The mechanism is under Update classes.

1 — Say which nodes must confirm

Every record starts as best_effort. Mark the nodes a release may not proceed without as required, and the nodes that must stay on a known build as left-behind; each edit lands in the authoring list, and one fleet sign covers them all:

$ atlasd fleet set-update-class gs1 required
$ atlasd fleet set-update-class relay-van required
$ atlasd fleet set-update-class survey-drone best-effort
$ atlasd fleet set-update-class test-bench left-behind --pinned
`test-bench`: update class left_behind, pinned (unsigned — run `atlasd fleet sign`, then push)
$ atlasd fleet sign
fleet `field-1` seq 12 signed by 3c24f045 (6 records, 5be2a1c0)
$ atlasd fleet push -c /etc/atlas/config.toml
fleet `field-1` seq 12 handed to the atlas0 daemon — it floods it over its sessions

A pinned left-behind node never takes a new epoch; without --pinned it takes the newest one left_behind_grace_secs (600 s by default) after it first sees it, which is after the watch is over. Either way it is excluded from targeting and from both gates, and the Fleet view shows it as running epoch N−k.

2 — Start the rollout with a release deadline

A rollout is two epochs: the policy to a canary set, then the same policy to everyone. The release now carries its own deadline. Leave release_watch_secs out to get three confirm windows plus two minutes, or set it per rollout:

$ curl -sS -X POST -H "Authorization: Bearer $TOKEN" -H "X-Atlas-CSRF: 1" -H "Content-Type: application/json" \
    -d '{"canary_nodes": ["9b1e..40aa"], "confirm_secs": 120, "release_watch_secs": 900}' \
    http://gs1:9800/api/config/rollout/start?interface=atlas0

Or set [config_mgmt] release_watch_secs on the pushing node as the default for every rollout it drives.

3 — Read both gates

$ atlasd fleet rollout status -c /etc/atlas/config.toml
rollout r-1791900000-9f3a1c2e · phase canary · canary epoch 13
  canary gate: 1 of 1 canaries applied and confirmed
    canary 9b1e..40aa   required     applied

$ atlasd fleet rollout release -c /etc/atlas/config.toml
rollout r-1791900000-9f3a1c2e released as epoch 14 (c0ffee12) — required nodes have until 1791900900 to confirm

The same two gates are in GET /api/config/rollout (gate and release_gate) and in the Fleet view's header, with a countdown to the release deadline and the required nodes still to confirm split out above everyone else.

4 — Let the gate decide

• Every required node confirmed → phase = "complete", journal rollout.complete. Best-effort nodes still on their way are listed under the gate and apply when they are next seen.
• A required node reports failed → an abort within one intake tick.
• A required node still unconfirmed at the deadline → an abort at the deadline.

$ atlasd fleet rollout status -c /etc/atlas/config.toml
rollout r-1791900000-9f3a1c2e · phase aborted · canary epoch 13
  canary gate: 1 of 1 canaries applied and confirmed
  aborted by: relay-van (4f02..c8d1) silent (release phase)
  release gate (epoch 14): required node relay-van (4f02..c8d1) did not confirm by the deadline for the released epoch 14 — the rollout aborts
    gs1            9b1e..40aa   required     applied            epoch 14   confirmed
    relay-van      4f02..c8d1   required     unknown            no epoch   SILENT
    drone-a        d1a0..7c21   best_effort  applied            epoch 14   confirmed
    drone-b        d1b4..0e93   best_effort  applied            epoch 14   confirmed
    test-bench     e77a..1b06   left_behind* left_behind        epoch N-1  held
    (* pinned: never takes a new epoch)

An abort is the pre-rollout policy signed forward as epoch 15 to everyone the release reached, so the fleet converges on the baseline. The cause is on the record as abort_cause (node_id_short, name, reason, phase), in the Fleet view, and in the journal as rollout.auto_abort with rollout_id, node, reason and phase. Until the watch ends, a new rollout is refused. An operator can also abort a released rollout by hand (atlasd fleet rollout abort --reason …) while it is still waiting on a required node.

5 — After an abort

Find the node: the Fleet view sorts it first, the topology overlay rings it, and atlasd fleet rollout status prints it. A node that was simply out of reach comes back on the baseline and needs nothing. A node that rejected the epoch has the reason in its own journal (epoch.rejected) — a version below min_daemon_version, a validation failure, a signature it does not trust. Fix the cause, then start the rollout again; the abort cost the fleet nothing but the time.

Boundaries

• The release gate reads the status nodes gossip back. A required node that is reachable but gossips nothing is silent, and silence aborts; make sure every required node has a gossip path to the pushing node.
• Classes are a property of the record, so changing one is a signed fleet-list change like any other, and takes effect for the next rollout.
• The gate waits for confirmation, not for a version: an epoch's min_daemon_version still decides at apply whether a node can take it at all.