feat(ops): device-audit script + canonical devices.yaml registry (#15) #16

Open

jmz wants to merge 1 commit from feat/15-device-audit into main

jmz commented

2026-05-28 00:10:54 +00:00

Owner

Summary

Closes #15. Adds devices.yaml (canonical bridge fleet) + scripts/audit-devices.py (diff against Synapse's per-user device list) + scheduled Forgejo workflow + tests + runbook update.

Closes a recurrence vector of #5: tonight's diagnostic dance for the mba-726dadit "ghost device" only proved the device was legitimate after manually finding its curve25519 in our local crypto store — pure luck. With this in place, the same investigation reduces to looking at a diff.

What's new

devices.yaml — canonical bridge-fleet registry, 4 Maximus/Felinus bridges pre-populated. Schema documented inline in the file.
scripts/audit-devices.py — stdlib-only (PyYAML is the sole third-party dep) script that diffs registry vs. Synapse and reports three classes of mismatch:
- UNEXPECTED: in Synapse, not in registry (rotation orphans, leaked tokens, forgotten test bridges)
- MISSING: in registry, not in Synapse (host died, was deleted, key purged)
- IDLE: registered but last_seen > idle_warn_days (default 30)
scripts/test_audit_devices.py — 16 stdlib unittest tests, no test-framework dep. Covers cross-signing-key filter, pure-logic audit() for each mismatch class + mixed state, YAML parsing + validation, and report-formatting sanity.
.forgejo/workflows/audit-devices.yml — weekly cron (Monday 06:00 UTC) + manual workflow_dispatch. Requires repo secret SYNAPSE_ADMIN_TOKEN.
docs/bridge-client-setup.md — new step 8 in the rotation procedure ("update devices.yaml in the same commit") + new "Auditing the device fleet" section.

Design choices

Pure-logic split: the audit() function is pure (no I/O); query_synapse_devices() is the only external interaction, called from main(). Tests exercise the pure parts; the HTTP path is left to manual CLI verification because mocking urllib for one endpoint is more ceremony than the call warrants.
Cross-signing-key filter: master/self-signing/user-signing keys sometimes appear in /devices responses depending on the homeserver. The filter looks for ids with / AND matching ^[A-Za-z0-9+/]{43}$ (the observed shape on @jmz and @kitty tonight). Real bridge ids (mba-…, nas-…, rpi5-…, plus pre-host-prefix 10-char ids like G9MCQri1N2) pass through.
never-seen ≠ idle: a device with last_seen_ts is None (registered but never connected) doesn't trigger the idle alert — it's a distinct state. If the device fails to materialize fully, the NEXT audit catches it via the missing-from-Synapse path.

Known blockers (not fixed by this PR)

This workflow won't dispatch until rpi5's self-hosted runner recovers. Tailscale ACL filtering broke rpi5's DNS tonight; the same blocker held up #14's CI run earlier. Separate handoff to Maximus@nas at nas:/root/handoff-headscale-acl-fix.md to fix the ACL. Once the runner is back, this workflow picks up on the next Monday 06:00 UTC.
SYNAPSE_ADMIN_TOKEN repo secret needs to be set in Settings → Secrets before the workflow can succeed. Token source: msc3861.admin_token in /opt/docker/matrix/files/homeserver.yaml on the NAS.

Test plan

python3 -m unittest scripts.test_audit_devices -v → 16/16
Script syntax-checks (ast.parse)
Spot-checked logic against the live fleet by inspection — current state is 3 Maximus bridges + 1 Felinus, all in Synapse, 0 unexpected, 0 missing, 0 idle → would report "OK".
(post-merge, after rpi5 runner recovers) workflow runs and reports "OK" on a clean fleet
(post-merge, periodically) workflow catches a real mismatch and fails loudly

🤖 Generated with Claude Code

## Summary Closes #15. Adds `devices.yaml` (canonical bridge fleet) + `scripts/audit-devices.py` (diff against Synapse's per-user device list) + scheduled Forgejo workflow + tests + runbook update. Closes a recurrence vector of #5: tonight's diagnostic dance for the `mba-726dadit` "ghost device" only proved the device was legitimate after manually finding its curve25519 in our local crypto store — pure luck. With this in place, the same investigation reduces to looking at a diff. ## What's new - **`devices.yaml`** — canonical bridge-fleet registry, 4 Maximus/Felinus bridges pre-populated. Schema documented inline in the file. - **`scripts/audit-devices.py`** — stdlib-only (PyYAML is the sole third-party dep) script that diffs registry vs. Synapse and reports three classes of mismatch: - **UNEXPECTED**: in Synapse, not in registry (rotation orphans, leaked tokens, forgotten test bridges) - **MISSING**: in registry, not in Synapse (host died, was deleted, key purged) - **IDLE**: registered but `last_seen` > `idle_warn_days` (default 30) - **`scripts/test_audit_devices.py`** — 16 stdlib `unittest` tests, no test-framework dep. Covers cross-signing-key filter, pure-logic `audit()` for each mismatch class + mixed state, YAML parsing + validation, and report-formatting sanity. - **`.forgejo/workflows/audit-devices.yml`** — weekly cron (Monday 06:00 UTC) + manual `workflow_dispatch`. Requires repo secret `SYNAPSE_ADMIN_TOKEN`. - **`docs/bridge-client-setup.md`** — new step 8 in the rotation procedure ("update `devices.yaml` in the same commit") + new "Auditing the device fleet" section. ## Design choices - **Pure-logic split**: the `audit()` function is pure (no I/O); `query_synapse_devices()` is the only external interaction, called from `main()`. Tests exercise the pure parts; the HTTP path is left to manual CLI verification because mocking `urllib` for one endpoint is more ceremony than the call warrants. - **Cross-signing-key filter**: master/self-signing/user-signing keys sometimes appear in `/devices` responses depending on the homeserver. The filter looks for ids with `/` AND matching `^[A-Za-z0-9+/]{43}$` (the observed shape on `@jmz` and `@kitty` tonight). Real bridge ids (`mba-…`, `nas-…`, `rpi5-…`, plus pre-host-prefix 10-char ids like `G9MCQri1N2`) pass through. - **`never-seen` ≠ `idle`**: a device with `last_seen_ts is None` (registered but never connected) doesn't trigger the idle alert — it's a distinct state. If the device fails to materialize fully, the NEXT audit catches it via the missing-from-Synapse path. ## Known blockers (not fixed by this PR) - **This workflow won't dispatch until rpi5's self-hosted runner recovers.** Tailscale ACL filtering broke rpi5's DNS tonight; the same blocker held up #14's CI run earlier. Separate handoff to Maximus@nas at `nas:/root/handoff-headscale-acl-fix.md` to fix the ACL. Once the runner is back, this workflow picks up on the next Monday 06:00 UTC. - **`SYNAPSE_ADMIN_TOKEN` repo secret** needs to be set in Settings → Secrets before the workflow can succeed. Token source: `msc3861.admin_token` in `/opt/docker/matrix/files/homeserver.yaml` on the NAS. ## Test plan - [x] `python3 -m unittest scripts.test_audit_devices -v` → 16/16 - [x] Script syntax-checks (`ast.parse`) - [x] Spot-checked logic against the live fleet by inspection — current state is 3 Maximus bridges + 1 Felinus, all in Synapse, 0 unexpected, 0 missing, 0 idle → would report "OK". - [ ] (post-merge, after rpi5 runner recovers) workflow runs and reports "OK" on a clean fleet - [ ] (post-merge, periodically) workflow catches a real mismatch and fails loudly 🤖 Generated with [Claude Code](https://claude.com/claude-code)

jmz added 1 commit

2026-05-28 00:10:54 +00:00

feat(ops): device-audit script + canonical devices.yaml registry (#15 )

check / check (pull_request) Successful in 3m4s

Details

ef46839dbc

Closes a recurrence vector of #5: the diagnostic dance that tonight
proved `mba-726dadit` was a live MBA bridge (not a ghost on Tabby's
Mac, as the issue had claimed) only worked because the device's
curve25519 was sitting in our local crypto store — pure luck. With
this in place the same investigation reduces to looking at a diff.

What's new
- `devices.yaml` (repo root): canonical bridge-fleet registry —
  4 Maximus/Felinus bridges pre-populated, plus an
  `user_devices_known` section snapshotting Element-client device
  ids per user as of tonight (for operator reference only, not
  audited).
- `scripts/audit-devices.py`: stdlib-only (PyYAML is the sole third-
  party dep) script that calls Synapse's
  `/_synapse/admin/v2/users/<mxid>/devices` for each registered
  mxid, diffs against the registry, reports three classes of
  mismatch (unexpected / missing / idle), and exits non-zero on
  ACTION REQUIRED.
- `scripts/test_audit_devices.py`: 16 stdlib `unittest` tests
  covering the cross-signing-key filter (real ids observed
  on @jmz and @kitty had the suspicious 43-char-with-slash shape
  Synapse leaks into device lists), the `audit()` pure-logic
  diff (clean / unexpected / missing / idle / never-seen / mixed),
  `load_registry` (YAML parsing + defaults + required-key
  validation), and `format_report` output sanity (header presence
  + OK/ACTION-REQUIRED switch).
- `.forgejo/workflows/audit-devices.yml`: weekly cron
  (Monday 06:00 UTC) + `workflow_dispatch` for manual. Installs
  PyYAML at job start, runs the script, fails the job on
  ACTION REQUIRED so the mismatch is visible in Forgejo's UI.
  Requires repo secret `SYNAPSE_ADMIN_TOKEN`.
- `docs/bridge-client-setup.md`: new step 8 in the rotation
  procedure ("update `devices.yaml` in the same commit") +
  new "Auditing the device fleet" section.

Design notes
- The script's audit logic is pure (no I/O at the audit-function
  level); the network call (`query_synapse_devices`) is the only
  external interaction, called from `main()`. Tests exercise the
  pure parts; the network path is left to manual CLI verification
  because mocking urllib for a single endpoint is more ceremony
  than the call warrants.
- Cross-signing keys (master / self-signing / user-signing) sometimes
  appear in `/devices` responses depending on the homeserver. The
  filter looks for ids with `/` AND matching `^[A-Za-z0-9+/]{43}$`
  (the observed shape on @jmz and @kitty tonight). Real bridge ids
  (`mba-…`, `nas-…`, `rpi5-…`, plus pre-rotation-runbook
  10-char auto-generated ids like `G9MCQri1N2`) pass through.
- `never-seen` (`last_seen_ts is None`) does NOT count as idle —
  it's a distinct state. If the device fails to materialize fully,
  the NEXT audit catches it via the missing-from-Synapse path.

Known blockers (not fixed by this commit)
- The `audit-devices.yml` workflow won't dispatch until the rpi5
  self-hosted runner recovers (Headscale ACL filtering breaks
  rpi5's DNS — separate handoff to Maximus@nas, see
  `nas:/root/handoff-headscale-acl-fix.md`). Once the runner is
  back, this workflow picks up on the next Monday 06:00 UTC.
- `SYNAPSE_ADMIN_TOKEN` repo secret needs to be set in
  Settings → Secrets before the workflow can succeed.

Tests
- `python3 -m unittest scripts.test_audit_devices -v` → 16/16
- Manual: spot-checked the audit against the live fleet —
  3 Maximus bridges + 1 Felinus all present in Synapse, 0
  unexpected, 0 missing, 0 idle, "OK".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>