Telemetry & observability

Documents the atlasd v0.2 release lineView as Markdown

A resilience product that cannot explain itself is a liability — "did it fail over?" must be answerable by machines during the event and by humans after it. Atlas exposes one coherent state through several doors: a snapshot file for scrapers, a socket for radios, a journal for the incident review, metrics for your NMS, and events for your SIEM. This page maps them and says which direction the data flows.

The stats snapshot — the integration point

Once per second the daemon rewrites /tmp/atlas/<iface>.json: every link's liveness, RTT, jitter, loss, capacity, quality score, byte counters and quota state; every peer's session state and per-link view; mesh membership; routes and topology (including advertised node names, roles and positions); crypto posture; and a "bonding tax" ledger of what the bonding layer itself costs in latency. The CLI, the dashboard, and the REST API are all readers of this same document — so anything you build on it sees exactly what the first-party tools see. GET /api/stats serves it over HTTP; the API reference shows the response shape.

The telemetry socket — what the OS cannot see

Kernel counters know nothing about a radio's world: RSSI, noise floor, remote-end signal, transmit-buffer headroom. Per-radio adapter processes bridge that gap over a Unix socket ([telemetry] enabled = true, path /run/atlas/telemetry.sock), speaking newline-delimited JSON. Two message kinds:

{"link_name": "uhf0", "ts": 1723550000.5, "source": "my-adapter",
 "rssi_dbm": -71, "noise_floor_dbm": -95, "tx_buffer_free_pct": 88,
 "extra": {"vendor_field": "anything"}}

{"node_position": {"lat_deg": 42.6, "lon_deg": 23.3, "alt_m": 590,
 "velocity": {"north_mps": 0.0, "east_mps": 4.2, "down_mps": 0.0}}}

link_name must match a [[link]] name; events for unknown links are dropped with a debug log. All radio fields are optional — send what your radio knows.
Position events feed topology and predictive scheduling; velocity is optional (the daemon derives it from consecutive fixes).
The daemon never depends on the socket: adapters can crash, restart and reconnect freely. Convention: a data-path bridge is named atlas-<radio>, a poll-only telemetry feeder atlas-telemetry-<radio>; both speak the same protocol. Writing an adapter for your own radio is an afternoon: read your radio's status interface, emit one JSON line per reading.

Fed RSSI shows up per link in the snapshot (radio_rssi_dbm, with source and age), on the dashboard — and, if enabled, in the RSSI-trend prediction that drains traffic off a fading link before it dies.

The event journal — answers after the fact

Live state forgets: once a link flaps back up, nothing in a gauge remembers 14:32. The persistent journal does (/var/lib/atlas/journal/<iface>.jsonl, on by default, size-capped at 4 × 2 MiB per interface with age-based expiry). Every entry carries a category (link, peer, config, mesh, route, …), a severity, a stable machine-readable code — link.down, peer.session_established, config.committed, link.quota_warn — and structured fields:

{"v":1, "seq":891, "ts":1723550001, "category":"link", "severity":"warn",
 "code":"link.down", "msg":"link uhf0 on nexus0 went down",
 "fields": {"reason":"probe_timeout", "missed_probes":5}}

Read it with atlasd journal -c link -s warn --since 24h --json or page it over GET /api/journal. Credential-shaped values are redacted at write time — the journal is safe to hand to a support engineer by construction.

Prometheus / OpenMetrics

GET /metrics is on by default ([metrics]), renders from the snapshots the daemon already writes, and costs nothing when nobody scrapes. Around fifty families cover links, peers, mesh, quotas and the QoS classes, in base units only — seconds, ratios, bits per second — with disciplined labels (interface, link, peer, class):

atlas_link_up{interface="nexus0",link="uhf0"} 1
atlas_link_rtt_seconds{interface="nexus0",link="uhf0"} 0.084
atlas_peer_last_handshake_seconds{interface="nexus0",peer="9Xp…"} 1.7235e9
atlas_link_quota_used_ratio{interface="nexus0",link="lte0"} 0.62

The monitoring guide has the scrape config and the alert rules worth starting from.

Syslog / CEF export opt-in · default off

For SIEM pipelines, [syslog] exports the same journal events as RFC 5424 structured syslog or ArcSight CEF over UDP, TCP or the local socket. The stable event codes double as the MSGID / signature id, so correlation rules survive upgrades. Delivery is deliberately non-blocking: a bounded queue drained by one writer thread — a dead collector can never stall the daemon, and dropped events are counted and reported into the stream once the collector returns.

Crash capture — local first, consent always

Crash reporting is designed for a no-phone-home posture: capture is on by default, sending is not. A crash produces a redacted report in a bounded local spool; it leaves the node only when an operator runs atlasd crash export (a zip for manual handling) or atlasd crash send — attributed to your console account, or anonymous, where anonymity also forces redaction. Uploaded reports strip addresses, endpoints and hostnames by default; the copy kept locally stays complete, because the operator reading their own node deserves the whole truth. A support engineer asking for diagnostics gets one honest artifact from atlasd support-bundle: config (secrets redacted), stats, health, journal tail, netcheck and crash spool, assembled by an allowlist so private keys can never leave by accident.

Hosted console reporting

A node explicitly enrolled into the hosted customer console self-reports per-link health roughly every 30 seconds ([telemetry] console_report, opt-out). A node that was never enrolled sends nothing meaningful — one lookup, one refusal, silence for the daemon's lifetime. Enrolment is always an operator action, never a default.