Newbury Exhibit v3

Fan-made shared multi-viewer AR experience for a physical LEGO® Hidden Side™ town display (tribute project — not affiliated with the LEGO Group). Multiple phones detect printed "Newbury Crests" (ArUco markers) and see the same server-driven ghosts anchored to real-world positions, each from their own viewpoint.

v3 = v2 with the tracking core replaced. Everything else — playlist engine, residents/tracks, character editor, admin suite, dev error overlay, password- protected final mode, deployment — carries over unchanged.

docs/PAGES.md — every page the exhibit serves, and the API endpoints worth opening while debugging. Keep it updated as pages change.

What changed in v3

The per-marker POS-IT → confidence-weighted fusion pipeline is replaced by a joint multi-marker board solve (public/js/ar/solve-cv.js, OpenCV.js):

Every detected crest corner becomes a 3D world point (from data/scene.json anchors, same anchorWorldMatrix convention as v2) and one solvePnP call recovers the camera pose. No per-marker fusion, no slerp base, no area buckets.
Stability scales with marker spread. Synthetic validation at 0.5px corner noise: ~0.06 cm position / 0.03° rotation error on the full 5-crest board (single-marker POS-IT on the same test: ~2.6 cm / 0.9°).
The wall crest makes the point set non-coplanar, which removes the POS-IT planar ambiguity (the flip problem) outright whenever it's visible.
Single visible crest still works: SQPNP/IPPE cold start, then iterative refine warm-started from the previous frame (temporal continuity resolves flips).
A reprojection-error gate (mean px, shown in the dev HUD) rejects poisoned frames before they reach the 1€ filter.

OpenCV.js (~13 MB WASM) is fetched at Docker build time (pinned @techstark/opencv-js@4.12.0-release.1) and lazy-loaded on the client after Start, with download progress shown in the dev HUD (cv:loading 42%). If it fails or stalls past 45 s, the app falls back to the v2 engine automatically.

Dual engines (on-table A/B)

The complete v2 pipeline (detect.js → pose.js → fuse.js) ships alongside the new one until the board solve is confirmed on the real table.

Dev HUD overlay tap zones (thirds):

zone	opencv engine	posit engine
top	switch engine (retries load if it failed)	switch engine
middle	axis mode (`std`/`ymirror`)	rotation mode (`fr`/…)
bottom	—	camera correction (`none`/…)

Engine choice persists in localStorage (nbx.engine). Optional per-device horizontal FOV override: localStorage nbx.hfovDeg (default 60, both engines).

axis ymirror exists as on-device insurance: if the world appears mirrored / camera under the table in opencv mode, tap the middle third once.

Reading the dev HUD

ws open  rx scene,active,pong        <- WebSocket state + recent message types
ghost recs 1                         <- ghost records the client is holding
eng opencv  FROZEN                   <- engine; FROZEN = ghost motion paused (tap tracking chip)
axis std  cv:ready                   <- engine mode + OpenCV load state
markers 2  reproj 0.4px              <- crests detected + mean reprojection error
raw / view / up / pos                <- solved camera pose
m0 px(0.02,-0.05) proj(-0.71,0.63)   <- AIM CHECK, see below
ghost ndc(-0.90,0.80) off-screen     <- first ghost's screen position

Aim check. px is where the crest actually sits in the camera frame (0,0 = centre); proj is where the solved pose projects it. Point straight at a crest so px ≈ (0,0) — if proj disagrees, the camera-frame convention is wrong, not the tracking. A sign flip on x means mirrored, on y means Y-inverted, on both means a 180° roll. Ghosts also carry a magenta always-on-top beacon in dev mode, so "no ghost" and "ghost off-screen" can't be confused.

First-time load (in place of v2)

git clone https://gitea.hideawaygaming.com.au/jessikitty/newbury-exhibit-v3.git ~/newbury-exhibit-v3
cd ~/newbury-exhibit-v3 && bash load-v3.sh

load-v3.sh imports the unchanged v2 base files into this repo (one commit, pushed — set GITEA_TOKEN=… if the machine has no git credentials), copies .env and any locally-dropped public/models from the v2 checkout, stops the v2 container, and brings v3 up on the same host port, container name, and exhibit-live volume — live scene/playlist/character state carries over.

Future deploys: cd ~/newbury-exhibit-v3 && bash deploy.sh (same as v2).
Rollback any time: cd ~/newbury-exhibit && docker compose up -d --build.

Conventions (unchanged from v2)

World units cm; origin front-left table corner; +X right, +Z back, +Y up.
Marker print sizes stay in mm (sizeMM); single mm→cm conversion at the solver boundary.
data/*.json are committed seeds; data/live/ (volume) is runtime state.
maxHammingDistance: 0 — hamming-1 detections are 100% phantoms here.

LEGO® and Hidden Side™ are trademarks of the LEGO Group, which does not sponsor, authorise or endorse this project.

5.2 KiB Raw Permalink Blame History