Fan-made shared multi-viewer AR experience for a physical LEGO® Hidden Side™ town display. Multiple phones point at printed ArUco markers ("Newbury Crests") and see the same server-driven ghosts anchored to real-world positions, each from their own viewpoint. Tribute project — not affiliated with or endorsed by the LEGO Group.

Stack

Node 20 (Express + ws) in Docker · Three.js r160 (CDN import map) · js-aruco2 (ARUCO_4X4_1000, vendored) · vanilla ES-module client.

Units

All world coordinates are centimetres (1 Three.js unit = 1 cm): anchor/spawn/building positions and sizes, wander radius, calibration readouts. Marker print size stays in mm (sizeMM); the mm→cm conversion happens once in fuse.js.

Baked-in AR fixes (hard-won, do not "simplify" these)

POS-IT pose: raw centered image coords into POS-IT (no Y pre-flip — pre-flipping double-inverts the vertical axis and makes tilt respond backwards); rotation used as-is; translation Y and Z negated (-t[1], -t[2]). A/B toggle: FLIP_INPUT_Y in public/js/ar/pose.js.
Planar-ambiguity flips: per-marker temporal consistency — pick the POS-IT solution closest to the previous frame (public/js/ar/pose.js).
Phantom marker IDs: maxHammingDistance: 0 (public/js/ar/detect.js).
Multi-marker flicker: confidence-weighted quaternion slerp fusion, hemisphere-aligned, weighted by marker screen area (public/js/ar/fuse.js). Place anchors densely so ≥2 crests are visible from any viewing angle.
Vendor load order matters: cv → svd → posit1 → aruco → dictionary.

Features

Playlist engine (server-driven, all viewers in sync): N concurrent ghosts, per-ghost dwell time then rotate through the whole roster, shuffle/roster order, color/rarity filters, optional time-of-day windows, crossfade in/out.
Behaviors: static float (bob + idle sway) or wander (deterministic orbit-drift — a pure function of server spawn record + wall clock, so every phone computes the same motion with zero position streaming). Wander is 3D: horizontal radius plus a configurable vertical amplitude (wanderVertical, cm).
Anchors ("Newbury Crests"): flat (horizontal) / wall (vertical) / custom mounts, fully editable — position, yaw/pitch/roll trim, printed size, enable toggle.
3D layout editor (/admin/layout.html): Three.js orbit view with drag gizmos (5 mm snap, green arrow = height) for anchors, occlusion buildings, and spawn points — spawn Y is fully editable so ghosts can live at any height. Anchors render through the exact same anchorWorldMatrix the AR pipeline uses, each with a white orientation arrow; compass rose on the grid (N = +Z = back of table). Top-view button, property panel, save/reset-to-seed.
Playlist manager (/admin/playlist.html), crest print sheet (/admin/print.html, true-to-size), calibration (/admin/calibrate.html: live distance/angle/jitter + fused pose), client error log (/admin/errors.html).
Occlusion: buildings render as invisible depth-only volumes so ghosts pass behind them.
Multi-part OBJ ghosts: legs/wisp + torso + head + headpiece assembled and tinted with the recovered Hidden Side gradient shader (Red/Angry, Yellow/Crazy, Blue/Sad). Procedural wisp fallback until models are dropped into public/models/ and listed in the model manifest (GET/PUT /api/models).

Modes

EXHIBIT_MODE=dev — admin pages open, on-screen client error overlay, errors reported to /api/client-errors.
EXHIBIT_MODE=final — admin + setup pages require ADMIN_PASSWORD (12 h tokens).

Run

cp .env.example .env       # set mode + password
docker compose up -d --build
# http://host:33044          → viewer
# http://host:33044/admin/layout.html → admin

Live edits (layout, playlist, models) persist in the exhibit-live volume and survive rebuilds; POST /api/scene/reset restores the committed seed.

Deploy on blackpearl

git clone https://gitea.hideawaygaming.com.au/jessikitty/newbury-exhibit-v2.git
cd newbury-exhibit-v2 && cp .env.example .env && bash deploy.sh

Then nginx-proxy 33044 with HTTPS (camera + sensors require a secure context).

Adding your OBJ ghost models

Drop parts into public/models/<id>/ (e.g. legs.obj, torso.obj, head.obj, headpiece.obj).
Register via the models API (or edit data/models.json seed):

{ "models": [ { "id": "minifig", "scale": 1.0, "parts": {
  "legs": "/models/minifig/legs.obj", "torso": "/models/minifig/torso.obj",
  "head": "/models/minifig/head.obj", "headpiece": "/models/minifig/headpiece.obj" } } ] }

The gradient shader recolors every part by the ghost's color type automatically.