Newbury Exhibit v2

Fan-made shared multi-viewer AR experience for a physical LEGO® Hidden Side™ town display. Multiple phones point at printed ArUco markers ("Newbury Crests") and see the same server-driven ghosts anchored to real-world positions, each from their own viewpoint. Tribute project — not affiliated with or endorsed by the LEGO Group.

Stack

Node 20 (Express + ws) in Docker · Three.js r160 (CDN import map) · js-aruco2 (ARUCO_4X4_1000, vendored) · vanilla ES-module client.

Units

All world coordinates are centimetres (1 Three.js unit = 1 cm): anchor/spawn/building positions and sizes, wander radius, calibration readouts. Marker print size stays in mm (sizeMM); the mm→cm conversion happens once in fuse.js.

Baked-in AR fixes (hard-won, do not "simplify" these)

POS-IT pose: raw centered image coords into POS-IT (no Y pre-flip — pre-flipping double-inverts the vertical axis and makes tilt respond backwards); rotation used as-is; translation Y and Z negated (-t[1], -t[2]). A/B toggle: FLIP_INPUT_Y in public/js/ar/pose.js.
Planar-ambiguity flips: per-marker temporal consistency — pick the POS-IT solution closest to the previous frame (public/js/ar/pose.js).
Phantom marker IDs: maxHammingDistance: 0 (public/js/ar/detect.js).
Multi-marker flicker: confidence-weighted quaternion slerp fusion, hemisphere-aligned, weighted by marker screen area (public/js/ar/fuse.js). Place anchors densely so ≥2 crests are visible from any viewing angle.
Vendor load order matters: cv → svd → posit1 → aruco → dictionary.

Features

Playlist engine (server-driven, all viewers in sync): N concurrent ghosts, per-ghost dwell time then rotate through the whole roster, shuffle/roster order, color/rarity filters, optional time-of-day windows, crossfade in/out.
Paths: waypoint routes (loop or pingpong) drawn in the 3D editor — ghosts walk them at constant speed and turn to face each new direction, like they live there. Waypoints are full 3D so paths can climb. Path motion is deterministic (pure function of time), same as wander.
Rarity system: weighted order makes Commons appear often and Legendaries rarely (per-rarity appearance weights), and movementScale shrinks how much rarer ghosts move.
Residents: pin any ghost permanently to a spawn or path (residents in the playlist); they never rotate out and their spot is excluded from rotation.
Behaviors: static float (bob + idle sway) or wander (deterministic orbit-drift — a pure function of server spawn record + wall clock, so every phone computes the same motion with zero position streaming). Wander is 3D: horizontal radius plus a configurable vertical amplitude (wanderVertical, cm).
Anchors ("Newbury Crests"): flat (horizontal) / wall (vertical) / custom mounts, fully editable — position, yaw/pitch/roll trim, printed size, enable toggle.
3D layout editor (/admin/layout.html): Three.js orbit view with drag gizmos (5 mm snap, green arrow = height) for anchors, occlusion buildings, and spawn points — spawn Y is fully editable so ghosts can live at any height. Anchors render through the exact same anchorWorldMatrix the AR pipeline uses, each with a white orientation arrow; compass rose on the grid (N = +Z = back of table). Top-view button, property panel, save/reset-to-seed.
Placement plan printer (/admin/plan.html): to-scale top-down plan of the whole layout — at 1:1 the actual crest patterns print true-size in position, so you can tape the sheets to the table and the markers are already placed. Tiles across A4/A3 with 10 mm overlap, crop marks, join labels (A1/B2…), overview page with tile map, per-layer toggles, grid ruler and compass. 1:2 / 1:5 / fit scales for checking proportions.
Playlist manager (/admin/playlist.html), crest print sheet (/admin/print.html, true-to-size), calibration (/admin/calibrate.html: live distance/angle/jitter + fused pose), client error log (/admin/errors.html).
Occlusion: buildings render as invisible depth-only volumes so ghosts pass behind them.
Multi-part OBJ ghosts: legs/wisp + torso + head + headpiece assembled and tinted with the recovered Hidden Side gradient shader (Red/Angry, Yellow/Crazy, Blue/Sad). Procedural wisp fallback until models are dropped into public/models/ and listed in the model manifest (GET/PUT /api/models).

Modes

EXHIBIT_MODE=dev — admin pages open, on-screen client error overlay, errors reported to /api/client-errors.
EXHIBIT_MODE=final — admin + setup pages require ADMIN_PASSWORD (12 h tokens).

Run

cp .env.example .env       # set mode + password
docker compose up -d --build
# http://host:33044          → viewer
# http://host:33044/admin/layout.html → admin

Live edits (layout, playlist, models) persist in the exhibit-live volume and survive rebuilds; POST /api/scene/reset restores the committed seed.

Deploy on blackpearl

git clone https://gitea.hideawaygaming.com.au/jessikitty/newbury-exhibit-v2.git
cd newbury-exhibit-v2 && cp .env.example .env && bash deploy.sh

Then nginx-proxy 33044 with HTTPS (camera + sensors require a secure context).

Adding your OBJ ghost models

Drop parts into public/models/<id>/ (e.g. legs.obj, torso.obj, head.obj, headpiece.obj).
Register via the models API (or edit data/models.json seed):

{ "models": [ { "id": "minifig", "scale": 1.0, "parts": {
  "legs": "/models/minifig/legs.obj", "torso": "/models/minifig/torso.obj",
  "head": "/models/minifig/head.obj", "headpiece": "/models/minifig/headpiece.obj" } } ] }

The gradient shader recolors every part by the ghost's color type automatically.

5.7 KiB Raw Blame History