A cellular fleet behind a web ground station shipped

Your aircraft carry two or three cellular modems, your users and aircraft live in your database, and your ground station is a web application. This guide puts one atlasd on each aircraft and one per customer on each of two hubs, admits aircraft from your database, removes them for good, and reads per-modem health into your own interface. Read Peer authentication and Identity and authorisation first if [auth] is new.

The plan

Each customer is its own Atlas network, with its own keys, mesh secret and address space; a captured aircraft exposes only its own customer's network. This guide builds customer 1 on 10.1.0.0/24. Every node of that network names its interface atlas-t1, because the fleet reads in step 6 ask each member for the interface they are called with. Customer 2 repeats everything on 10.2.0.0/24, with the interface atlas-t2 and ports 42001–42003.

Every aircraft and the backend dial out, so carrier NAT needs no configuration and nothing on the aircraft listens on a public address. Each hub link meets one modem slot: the aircraft's first modem dials port 41001 on both hubs, its second 41002, a third 41003. Commands from the backend reach an aircraft through either hub, and the aircraft keeps one tunnel address through modem re-attaches, a hub going away and key rotation.

1 — The hubs

Run one daemon per customer on each hub, each with its own config, TUN device and ports (see Deploying with systemd for the unit). The two hubs of a customer peer with each other, so routes and the fleet list flood across both.

[interface]
# the same interface name on every node of customer 1
name        = "atlas-t1"
address     = "10.1.0.1/24"
private_key = "…"
node_name   = "hub-a"

[[link]]            # every aircraft's first modem
name         = "m1"
bind_device  = "eth0"
bind_address = "0.0.0.0"
listen_port  = 41001

[[link]]            # the second modem
name         = "m2"
bind_device  = "eth0"
bind_address = "0.0.0.0"
listen_port  = 41002

[[link]]            # a third modem
name         = "m3"
bind_device  = "eth0"
bind_address = "0.0.0.0"
listen_port  = 41003

[[peer]]            # hub B, customer 1
public_key        = "<hub B public key>"
allowed_ips       = ["10.1.0.2/32"]
endpoint_per_link = { "m1" = "203.0.113.20:41001" }

[mesh]
enabled     = true
# openssl rand -base64 32; the same on every node of customer 1
mesh_secret = "<customer 1 secret>"

[qos]
enabled = true

[traversal]
# lets m2 and m3 learn each modem's address (below)
enabled = true

Hub B is the same file with address = "10.1.0.2/24", its own key and hub A as the peer at 198.51.100.10:41001. Aircraft and the backend are not listed: they are admitted in step 4.

Why [traversal] enabled. A node admitted at handshake time is reached, at first, only on the hub link its handshake arrived on. With traversal on, each other hub link takes the address that modem's first authenticated packet came from, and endpoint roaming follows it when the carrier re-addresses the modem. Without it, the hub would answer an aircraft over one modem only.

Serve each daemon's API on its own tunnel address, so it is reachable only from inside the customer's network:

atlasd web --interface atlas-t1 --listen 10.1.0.1:9800

2 — The aircraft

One [[link]] per modem, each bound to the modem's interface, with a declared capacity and, on a metered SIM, a quota. The private key is generated on the aircraft with atlasd keygen and never leaves it; only the public half travels.

[interface]
name        = "atlas-t1"
address     = "10.1.0.17/24"
# generated here, never copied off the aircraft
private_key = "…"

[[link]]
name        = "wwan0"
bind_device = "wwan0"
listen_port = 51820
capacity    = "20 Mbps"

[[link]]
name            = "wwan1"
bind_device     = "wwan1"
listen_port     = 51821
capacity        = "20 Mbps"
quota           = "20 GB"    # the metered SIM
quota_period    = "monthly"
quota_reset_day = 1

[[peer]]            # hub A
public_key           = "<hub A public key>"
allowed_ips          = ["10.1.0.1/32"]
# keeps each modem's NAT mapping open
persistent_keepalive = 20
endpoint_per_link    = { "wwan0" = "198.51.100.10:41001", "wwan1" = "198.51.100.10:41002" }

[[peer]]            # hub B
public_key           = "<hub B public key>"
allowed_ips          = ["10.1.0.2/32"]
persistent_keepalive = 20
endpoint_per_link    = { "wwan0" = "203.0.113.20:41001", "wwan1" = "203.0.113.20:41002" }

[mesh]
# routes to the backend are learned from the hubs
enabled     = true
mesh_secret = "<customer 1 secret>"

[qos]
enabled = true

• endpoint_per_link ties each modem to its hub port by name, so a change in the order the modems enumerate cannot aim a link at the wrong port.
• Every link has its own listen_port. Two links on one device and address must never share a port; the reason is on Known limitations.
• The backend is configured like an aircraft with one link: lan on its own interface, the two hubs as peers at …:41001 with persistent_keepalive, the same [mesh] and [qos] enabled = true.

Check every file before anything starts:

atlasd config check -c /etc/atlas/atlas-t1.toml

3 — Classify the flight stack's traffic

The flight stack needs no change. The classifier reads DSCP first, then ports:

Control is copied on the two best modems at once and the receiver drops the second copy; on a single live modem each Control packet is repeated in time instead. Video and logs queue behind Control in their own classes, and Bulk is sent only when nothing live is waiting and is held through an outage. The policy behind each class is on Transport classes.

Commands toward the aircraft are classified on the backend, where they enter the tunnel. The port rule matches the destination port, so send them to UDP 14550 on the aircraft, mark them CS6, or add a rule on the backend for the port your MAVLink router listens on:

[[qos.flow]]
# the aircraft router's port, when it is not 14550/14551
dst_port = 14555
class    = "Control"

Keep video off the metered SIM with a link group. Ungrouped links form the group default, so the unmetered modem still carries everything:

[[link_group]]
name  = "metered"
links = ["wwan1"]

[qos.group.metered.Video]
allowed = false

With QoS on, the low band — Video, Telemetry and Bulk — is paced to each modem's measured drain rate, so a video fragment does not sit in the modem's queue in front of the next command. The keys are low_band_pacing and low_band_target_ms.

4 — Admit aircraft from your database

Pick one of four ways: a command hook into your database, records your backend writes, a pairing code the ground station shows, or a certificate signed at your factory. All four keep your database the system of record for which user owns which aircraft; Atlas holds only keys, addresses and, with records, permissions. None admits an unknown key silently.

A command hook on each hub

The hub runs your hook when a handshake arrives from a key it does not know. Add the section to each customer daemon on both hubs:

[auth]
provider = "command"
command  = "/usr/local/lib/atlas/gcs-authorize"
command_timeout_ms = 3000
deny_cache_secs    = 30

The hook gets the base64 public key and the source address as arguments (and as ATLAS_PEER_PUBLIC_KEY and ATLAS_PEER_SOURCE). Exit 0 admits the key and every non-empty stdout line is an allowed-IP CIDR; any other exit denies. One row per node — aircraft and the backend alike — holds its public key, customer, tunnel address and status:

#!/bin/sh
# gcs-authorize <base64-pubkey> <source-addr>
# GCS_DB, set in the daemon's unit: a read-only role
# on a database both hubs can reach.
ip=$(psql "$GCS_DB" -qtA -v key="$1" <<'SQL'
select tunnel_ip from drones
 where public_key = :'key' and tenant_id = 1 and status = 'active';
SQL
) || exit 1
[ -n "$ip" ] || exit 1
echo "$ip/32"

#!/bin/sh
# Your endpoint answers 200 with "10.1.0.17/32",
# or 404 for a key it does not admit.
curl -fsS --max-time 2 -G "https://gcs.example.internal/atlas/authorize" \
     --data-urlencode "key=$1" --data-urlencode "tenant=1" || exit 1

• Every failure fails closed. A hook that runs past command_timeout_ms is stopped and the handshake denied; a non-zero exit denies; a grant with no usable CIDR is a deny; a hook that cannot be started denies every key; provider = "command" without command keeps the daemon from starting.
• Denials are cached for deny_cache_secs, so a handshake flood never forks one process per packet. A row you activate right after a denial takes effect when that window ends.
• Admissions are not re-asked. An admitted key becomes a peer of the hub and the hook is not run for it again; removal is step 5.
• A hook grants addresses only: an aircraft admitted this way holds every capability. For per-aircraft permissions, use records.

Records in a signed fleet list

With [auth] provider = "fleet" on every node of the customer, each node is a signed record with its own capability flags. On each aircraft, atlasd keygen --identity mints the record UUID and both keypairs, prints the node's [interface] lines and the matching fleet add-node line for the operator. Your backend, holding the customer's config-author key, writes the records:

$ atlasd fleet init --fleet tenant-1 --file /etc/atlas/tenant-1-fleet.toml

$ atlasd fleet add-node --file /etc/atlas/tenant-1-fleet.toml --name hub-a \
    --node-id <uuid> --transport-key <pub> --signing-key <pub> \
    --allowed-ip 10.1.0.1/32 --role hub --relay \
    --classes Control,Voice,Position,Telemetry,Video,Bulk
# hub-b the same, with 10.1.0.2/32

$ atlasd fleet add-node --file /etc/atlas/tenant-1-fleet.toml --name gcs \
    --node-id <uuid> --transport-key <pub> --signing-key <pub> \
    --allowed-ip 10.1.0.10/32 --role operator \
    --classes Control,Position,Telemetry,Video,Bulk --dashboard fleet

$ atlasd fleet add-node --file /etc/atlas/tenant-1-fleet.toml --name drone-17 \
    --node-id <uuid> --transport-key <pub> --signing-key <pub> \
    --allowed-ip 10.1.0.17/32 --role uav \
    --classes Control,Position,Telemetry,Video,Bulk

$ atlasd fleet check --file /etc/atlas/tenant-1-fleet.toml
$ atlasd fleet sign  --file /etc/atlas/tenant-1-fleet.toml
$ atlasd fleet push  --file /etc/atlas/tenant-1-fleet.toml -c /etc/atlas/atlas-t1.toml

• Hubs carry --relay, because they forward between the aircraft and the backend. Aircraft keep the default relay = false: no traffic is ever routed through an aircraft.
• Every flying aircraft gets Control explicitly. A record without --classes holds Telemetry, Position and Bulk, and a Control-marked packet from it is sent as Position. atlasd fleet set-caps drone-17 --classes … changes it later; sign and push again.
• The backend reads the whole fleet with --dashboard fleet; aircraft keep own.
• fleet push hands the signed list to the backend's own daemon, which floods it over its sessions to both hubs and every aircraft. A new aircraft needs the signed list that holds its record as its bootstrap [auth] fleet_list for its first start; later lists reach it over its sessions. Every node lists the author's public key in its config-authors.toml.

[interface]
node_id     = "<record uuid>"
signing_key = "…"

[auth]
provider   = "fleet"
# bootstrap copy
fleet_list = "/etc/atlas/tenant-1-fleet.toml"

A pairing code from the ground station

With records in place, “Add drone” needs no signing host. Give one hub a record that may sign the customer's list — add --config-author to its fleet add-node line above — and let the backend ask that hub for a pairing code with a write token:

$ curl -X POST -H "Authorization: Bearer $HUB_TOKEN" -H "X-Atlas-CSRF: 1" \
       -H "Content-Type: application/json" \
       -d '{"name": "drone-17", "allowed_ips": ["10.1.0.17/32"], "ttl_secs": 900}' \
       http://10.1.0.1:9800/api/admission/invites?interface=atlas-t1
{ "ok": true, "id": "…", "uri": "atlas:pair?v=3&k=…&n=drone-17&a=…&i=10.1.0.17/32&h=10.1.0.1/24&t=…",
  "code": { "version": 3, "anchors": ["…"], "address": "10.1.0.17/32", "hub_address": "10.1.0.1/24", … }, "expires_at": …, "can_sign": true, … }

Show uri as a QR. The aircraft starts with its identity from atlasd keygen --identity, both hubs as peers, and the code in place of the fleet list:

[auth]
provider     = "fleet"
invite_token = "atlas:pair?v=3&k=…&n=drone-17&a=…&i=10.1.0.17/32&h=10.1.0.1/24&t=…"   # the whole code

• One code, one aircraft, once. Hub A spends the code on the aircraft's first handshake, mints drone-17 with the least it needs (no relaying, Telemetry, Position, Bulk), signs the next list and floods it; hub B admits the aircraft on its next handshake. The same code from a second key gets no session.
• Give flying aircraft Control in the invitation: add "classes": ["Control", "Position", "Telemetry", "Video", "Bulk"] to the body.
• The aircraft trusts hub A's list: the code carries the fleet's anchors (a=) and both tunnel addresses (i=, h=) under the token's MAC, so the aircraft adopts the first list signed by one of those anchors, or by hub A's own config-author record, once hub A has confirmed the code over the session — then enforces records like every node. Its [interface] address may be left out: it takes the code's. See what the code carries.
• Every option, the CLI form and what each step journals are in Admit nodes by pairing code, certificate or approval.

A certificate from your factory

For airframes you build, sign each one's record once with the customer's authority key and ship the certificate on the aircraft; neither hub needs an entry for it:

$ atlasd fleet cert --name drone-17 --node-id <uuid> --transport-key <pub> --signing-key <pub> \
    --allowed-ip 10.1.0.17/32 --classes Control,Position,Telemetry,Video,Bulk --fleet tenant-1 \
    --key /secure/tenant-1-authority.key --out drone-17.cert

On the aircraft, [auth] provider = "fleet" and cert_file = "/etc/atlas/drone-17.cert". The authority's public key goes in each hub's and the backend's config-authors.toml. A hub admits the aircraft with the certificate's flags and offers the certificate on, so the other hub and the backend learn the record and reach it. The claim code printed on the airframe stays in your application, which links the aircraft to the user's account.

5 — Remove an aircraft

With the hook, set the row inactive: from then on the hook refuses every new handshake from that key. A session a hub already holds is not re-checked, so end it on both hubs; the aircraft's next handshake runs the hook and is refused.

# on hub A and on hub B
$ atlasd peer evict "<base64 public key>" --reason "sold" -c /etc/atlas/atlas-t1.toml

With records, revoke the record on the backend and push in the same command:

$ atlasd fleet revoke drone-17 --reason "sold" \
    --file /etc/atlas/tenant-1-fleet.toml --push /etc/atlas/atlas-t1.toml
  removed `drone-17` (3f1c..9a07) — id and keys revoked
  mesh secret rotated (5b2e90d1), sealed to 3 remaining record(s)
fleet `tenant-1` seq 6 signed by 3c24f045 (3 records, 7d41c2aa)
fleet `tenant-1` seq 6 handed to the atlas-t1 daemon — it floods it over its sessions

• Every node that adopts the list evicts the aircraft's sessions in the same intake, purges its adverts and roster entry, and switches to the rotated mesh secret, which the removed aircraft does not hold. Its retried handshakes are refused on both hubs.
• Each node's journal records security.revoked and security.peer_evicted; atlasd fleet show --revoked lists the revocations.
• An aircraft admitted by a pairing code is a record like any other. One admitted by certificate has no list entry: revoke the certificate with atlasd fleet revoke --cert drone-17.cert, and it is refused from then on, even from a factory-fresh start.
• A hub that minted records itself has a newer list than your authoring file: atlasd fleet pull -c <hub config> on the backend brings it back before the next edit.
• On the removed aircraft, atlasd status says it is revoked and that no peer will accept it.
• A revoked id or key never returns in a later list. An aircraft that comes back is enrolled again as a new identity, with a new UUID and new keys.
• Every flag of fleet revoke and peer evict is in the CLI reference; checking that every node holds the removal is in Remove a compromised node.

6 — Read per-modem health into your interface

Each aircraft measures its own modems — every [[link]] is one modem, probed every 250 ms. Serve its API on its tunnel address and read it from the backend over the tunnel:

# on the aircraft
atlasd web --interface atlas-t1 --listen 10.1.0.17:9800

# on the backend
$ curl -s "http://10.1.0.17:9800/api/stats?interface=atlas-t1" \
    | jq '.links[] | {name, alive, rtt_ms, jitter_ms, loss_pct, quality_score, quota}'
{ "name": "wwan0", "alive": true, "rtt_ms": 48.2, "jitter_ms": 3.1, "loss_pct": 0.0,
  "quality_score": 0.93, "quota": null }
{ "name": "wwan1", "alive": true, "rtt_ms": 61.7, "jitter_ms": 5.4, "loss_pct": 0.4,
  "quality_score": 0.88, "quota": { "used_bytes": 7340032000, "budget_bytes": 20000000000,
  "used_ratio": 0.367, "period": "monthly", "warning": false, "over_budget": false, … } }

Credentials. On a node running [web] protect = "all", reads need a credential:

• Node-bound tokens in a record fleet: atlasd config node-token -c /etc/atlas/atlas-t1.toml on the backend mints a token for the X-Atlas-Node-Token header, valid for a five-minute window. The backend's dashboard = "fleet" lets it read any member; an aircraft's own reads only itself.
• Named API tokens for your services: atlasd web token add gcs --scope read --expires 30d on a hub prints a bearer token once; atlasd web token revoke gcs ends it. Tokens belong to the host, so one token reads every customer daemon on that hub, and they need at least one dashboard account on it.

Write authority — configuration pushes and fleet lists — is an Ed25519 signature from an author key and is never granted by a read token. Alert rules for these metrics are in Monitoring integration; every response shape is on the REST API page.

7 — Prove it

The edges to design around — the size of one fleet list, hop-by-hop forwarding through the hubs, one protocol version across the fleet — are on Known limitations.