docs: architecture record — board solve, worker offload, camera model, pose validity, scene flow

.claude-context/01-architecture.md

@@ -0,0 +1,121 @@
+# Architecture
+
+Current as of 2026-09-23. Supersedes the v2 tracking design throughout.
+
+## Pipeline
+
+```
+camera video
+  -> detection canvas (long side capped, ~20Hz)
+  -> ArUco decode, maxHamming 0, filtered to scene anchor ids, deduped by id
+  -> TRACKING ENGINE  ->  camera pose in world
+  -> three.js render: server-driven ghosts + depth-only occluder meshes
+```
+
+Rendering runs every animation frame; detection and solving do not.
+
+## Tracking engine
+
+**Primary: `opencv`** — a joint multi-marker *board* solve. Every detected crest
+corner becomes a 3D world point (from scene anchors via `anchorWorldMatrix`), and
+**one `solvePnP`** recovers the camera pose. This replaced v2's per-marker POS-IT
+plus confidence-weighted fusion: stability now scales with marker spread instead
+of degrading into fusion tuning, and there is no per-marker slerp base or area
+bucketing.
+
+**Fallback: `posit`** — the complete v2 pipeline (`detect` → `pose` → `fuse`),
+retained for on-table A/B and as an automatic fallback if the OpenCV worker fails
+to load. To be deleted once opencv is confirmed on the real town.
+
+Engine is runtime-switchable from the dev HUD and persists in
+`localStorage.nbx.engine`.
+
+### Why OpenCV runs in a Web Worker
+
+The `@techstark/opencv-js` build embeds ~8 MB of WASM as a base64 data URI inside
+a 10.8 MB script. Parsing, base64-decoding and compiling that on the main thread
+froze the page for seconds and triggered "page unresponsive" on mobile. In
+`cv-worker.js` the whole cost — and every per-frame `solvePnP` — is off-thread.
+
+Solves are **fire-and-forget, one in flight**: if a solve is still running the
+frame is skipped rather than queued, so the main thread never blocks and latency
+can't accumulate. The 1€ filter (main thread) absorbs the ~1 frame delay.
+
+Pinned to `4.12.0-release.1`. Note for future work: this build lacks
+`solvePnPGeneric` and `cornerSubPix`, which is why pose ambiguity is handled by
+gating rather than by comparing both IPPE solutions.
+
+## Camera model
+
+Everything derives from one assumption — `HFOV_DEG` (default 60, override via
+`localStorage.nbx.hfovDeg`) — **applied to the image's LONG axis**, because iOS
+may hand back a portrait stream (720×1280). Keying it to `videoWidth` made the
+focal length 1.78× too short.
+
+```js
+focalPxFor(w, h) = (max(w, h) / 2) / tan(HFOV_DEG / 2)
+```
+
+The same function feeds both the detector and the render camera, so they cannot
+disagree. The render camera additionally accounts for `object-fit: cover`: only
+part of the frame is on screen, so `camera3.fov` comes from the *visible* height,
+and `_crop` records the visible fraction so on-screen diagnostics compare like
+with like.
+
+**Frame conversion** (validated against synthetic projections, error ~0):
+solvePnP returns world→cvCamera `(R, t)`; cv camera is +X right, +Y down,
++Z forward. Camera position `C = -Rᵀt`; three.js world-from-camera rotation is
+`Rᵀ · diag(1, -1, -1)`.
+
+## Pose validity: the coplanar ambiguity
+
+**A coplanar set of markers admits two poses that reproject almost equally well**
+— the true one and a mirrored twin. With few crests visible the solver
+intermittently returns the twin, which looks like ghosts snapping to the wrong
+side of the table. Reprojection error cannot separate them: a flipped pose was
+observed accepted at 1.3 px.
+
+Two independent measures, in order of preference:
+
+1. **Geometry (the cure).** Non-coplanar anchors make the twin mathematically
+   impossible. The A3 test rig provides this with a folded tower carrying crests
+   100 mm above the sheet; the real town should keep at least one raised or
+   wall-mounted crest for the same reason.
+2. **Physics gates (the safety net),** in `cv-worker.js`: the camera must be
+   above the table, and cannot teleport 20 cm between frames. A rejected pose
+   leaves the previous one in place. An escape hatch after 12 consecutive
+   rejections prevents a bad reference locking tracking out. The HUD reports the
+   rejection reason.
+
+Detection also **deduplicates ids within a frame**, keeping the largest quad:
+two quads decoding to one id would hand solvePnP contradictory image points for
+identical world points.
+
+## Scene and state flow
+
+- **Scene is fetched over HTTP at startup**, not awaited from the WebSocket.
+  Detection filters against scene anchor ids, so a slow or flapping socket used
+  to make every detection get silently discarded — indistinguishable on screen
+  from "the camera sees nothing". The WebSocket `scene` message now only
+  refreshes an already-working client.
+- The WebSocket remains authoritative for **ghosts** (`spawn`, `despawn`,
+  `active`, `characters`) and clock sync (`ping`/`pong`).
+- **Motion amounts are server-authoritative.** `playlist.js` sends explicit
+  `bobAmp`, `radius` and `speed` with every spawn record, so client-side defaults
+  never apply to a real ghost. Tune motion in the playlist config.
+
+## Occlusion
+
+Invisible depth-only meshes (`colorWrite: false`) let physical models hide
+ghosts. Two shapes: boxes for buildings, and cylinders (`shape:'cylinder'` with
+`radiusCm`/`heightCm`) for round props, where a box silhouette would wrongly hide
+ghosts at the corners. Positions are floor-level; meshes are lifted by half their
+height.
+
+## Caching
+
+App JS and HTML are served `no-cache` (ETag makes revalidation cheap); the 13 MB
+`opencv.js` is `immutable, max-age=1yr`. A cache-busting query on the entry module
+would not work — ES module imports fetch under their own URLs, so children would
+stay stale. Note that an upstream openresty proxy has been seen overriding this
+with a daily `expires`; that needs a vhost fix.