Orrery

PRD-017 · Sensory Layer — Gyroscope, Sonification & Haptics ​

Status · Draft v0.4 (all v1 architectural decisions resolved 2026-05-16) Date · 2026-05-16 Owner · Marko Closes into · RFC-020 Slice gate · v1.x (after PRD-015 mobile + PRD-016 audio narration ship)

Why this is a PRD. Mapping Orrery's continuous physics streams (orbital velocity, distance, signal delay, regime, fuel, microgravity) onto three additional output channels — gyroscope (input camera control), Web Audio sonification, Vibration / Taptic haptics — turns the device into a physical instrument for understanding orbital mechanics. The decision touches the audio channel (must coexist with PRD-016 narration), the mobile wrapper (uses @capacitor/haptics from PRD-015's plugin list), the accessibility surface (must respect prefers-reduced-motion + screen readers), and every 3D scene's camera controller (gyro input pauses when touch is active). It needs a product gate before any sensor permission, audio context, or vibration call lands.

§why ​

The original orrery — the 18th-century brass instrument the product is named after — was a thing you held. You turned the handle. The gears clicked. The metal made sound. The motion was in your hands.

Orrery the web app has rebuilt the visual language of that instrument across 11 routes and 7 3D scenes. What it has not rebuilt is the embodied half: the input you give the instrument with your hand, the sound the instrument makes back, the small physical confirmation that something happened.

Mobile devices have all three of those affordances already wired in. The gyroscope reads the rotation of the device; tilting becomes camera control. The Web Audio API turns any numerical stream into pitch, gain, filter, reverb. The Vibration API (Android) and Taptic Engine (iOS, via Capacitor) produce tactile pulses. Every frame, Orrery's physics already computes the streams: heliocentric velocity, distance from Earth, fuel remaining, orbital regime, signal delay. The data exists. The hardware exists. The plumbing between them is the feature.

This is not ornamental. Sonification has a specific educational advantage over visual display: it conveys change over time more intuitively than numbers. A HUD reading "31.4 km/s" at one instant tells you a number. A tone that rises and falls over the arc of a Mars mission conveys acceleration and deceleration continuously, without requiring the user to watch a specific spot on the screen. The user tilts their phone to look at Mars, hears the velocity tone rising, feels a discrete pulse when the spacecraft crosses Mars's orbit — three senses confirming the same physical event. That is how humans learn best.

Kepler knew this. In 1619, he computed the ratios of planetary orbital velocities at perihelion and aphelion and noticed they formed approximate musical intervals. He published them in Harmonices Mundi — the harmony of the worlds. Orrery makes that harmony literal: each planet gets an oscillator tuned to its angular velocity. The solar system has a chord.

§audiences ​

AudienceWhy the sensory layer helps them
Curious learnerTilting the phone to orbit is more discoverable than learning two-finger drag-and-pinch. The sound makes the physics audible without reading numbers.
STEM studentDiscrete tones for orbital regimes, agency-coloured timbres for spacecraft families — sonification is a study aid that doesn't compete with the visual.
Educator (classroom / kiosk / museum)A device on a table with a kid tilting it and hearing the planets is an entirely different educational artefact than a static screen. The kit becomes the lesson.
Vision-impaired userA planet-selection haptic confirms the touch happened; the sonic identity of /explore (Kepler chord) means the route is identifiable by sound. Captions (PRD-016) supply narration; sensory layer supplies event confirmation.
Power user (any audience)Master toggle off by default — opt-in for everyone. Once enabled, the layer disappears into the experience.

§what's already shipped (sensory-readiness inventory) ​

CapabilityStatusSource
11 primary nav routes (7 with 3D scenes)shipped (v0.6)TA.md route inventory
Camera controllers per 3D sceneshippedsrc/lib/camera-*.ts modules
prefers-reduced-motion respected throughoutshippedADR-022, ADR-025
Mobile-first layout (bottom-bar, bottom-sheet patterns)shipped (v0.6)ADR-018
Capacitor wrapper with @capacitor/haptics pluginplanned (PRD-015 / RFC-018, v0.8)RFC-018 §6 plugin list
Audio narration overlay (PRD-016)planned (v0.9)will share audio channel — see §audio-coexistence below
Cookie-only persistence (orrery_locale)shippedADR-057 — sensory uses in-memory + sessionStorage only

§goal ​

Ship a sensory layer that reaches across all 11 routes by v1.x (after PRD-015 mobile + PRD-016 audio land):

  • Gyroscope: mobile-only camera input on the 7 3D-scene routes (/explore, /fly heliocentric, /fly cislunar, /earth, /moon, /mars, /iss, /tiangong).
  • Sonification: desktop + mobile, all 11 routes (per-route specialisation per RFC-020 §sonification).
  • Haptics: discrete pulse events on Android (web navigator.vibrate) and on both Android + iOS via @capacitor/haptics when running under the Capacitor wrapper.

The original draft scoped sonification to 6 prototype screens (P01–P06). The v0.6.0 reality is 11 routes and Marko wants editorial parity across all of them.

§promises (v1) ​

  1. /explore plays a Kepler chord when AUDIO is on — eight planet oscillators tuned to angular velocity, sine + triangle waveforms, spatial panning that follows camera orientation. The solar system sounds harmonic.
  2. /fly mission arcs are audible. Velocity → pitch (the protagonist), distance → reverb depth (the isolation), fuel → harmonic distortion (the mortality), signal-delay → echo (the CAPCOM wait). Arrival = major chord.
  3. /fly porkchop topology is heard, not just seen. Cell-tap plays a tone whose pitch maps to Δv. Drag across the magnifier is continuous sonification of the topology you're crossing.
  4. /missions has agency tones. Card selection = agency-coloured chord (NASA warm, ESA bright, JAXA crisp, ROSCOSMOS low, CNSA bright, ISRO mid). Status modulates: ACTIVE = oscillation, FLOWN = clean, PLANNED = soft echo.
  5. /earth orbit regimes are drone-mapped. LEO = sine, MEO = triangle, GEO = square, HEO = sawtooth, L-points = sine + detune. Selected object boosts its regime drone.
  6. /moon has a continuous filter sweep. Near-side = warm low-pass, far-side = bright. Landing-site selection plays the agency tone of the operator.
  7. /mars has the same regime + landing-site palette as /earth + /moon, adapted: distance-from-Earth modulates a global filter (closer = warmer); rover-position selection plays the operator's agency tone.
  8. /iss modules sing. Each module gets a tone family by function (habitation = warm low, lab = mid bright, docking = pulsing). Visitor presence (Soyuz / Crew Dragon docked) adds a subtle additional drone. Selected module boosts its tone.
  9. /tiangong uses the same instrument class as /iss but a brighter palette (triangle + sawtooth instead of sine + triangle). Different agency = different "voice" of the same module-tone instrument.
  10. /science shifts ambient texture per chapter. life-in-space = warm pad, observation = airy chime texture, mission-phases = rhythmic pulse, basics = a cleaner sustained tone. Discrete chapter switch is a soft transition.
  11. /fleet has spacecraft-family chords. Selecting a spacecraft = 3-note chord in the agency's tone palette. ACTIVE status = continuous low ambient drone; FLOWN = clean single tone; PLANNED = soft echoey hint.
  12. / (landing) plays a soft Curator-tour intro tone when AUDIO is enabled — only at the very top of the page, not on every navigation back.
  13. Haptic threshold events. Mission arc crosses Mars orbit → 1 pulse. Δv warning → 1 pulse. Fuel exhaustion → triple pulse. Module / planet / site / spacecraft selection → short single pulse. Solver-complete → triple pulse. 12 patterns total per RFC-020 §haptics.
  14. Gyroscope camera control on the 7 3D-scene routes. Reference-frame calibrated (G-C from the original RFC). Touch input pauses gyro (T-B). Recalibrate gesture re-anchors the home position.
  15. One master toggle (off by default), one settings sheet in the right of the nav. Long-press on the master toggle opens a sheet with sub-toggles for GYRO / AUDIO / HAPTIC. Desktop hides GYRO + HAPTIC (only AUDIO is meaningful). prefers-reduced-motion users get only AUDIO surfaced.

§scope ​

v1 (in scope) ​

  • All 11 routes get sonification (per-route specialisation; see RFC-020 §sonification details for the 5 new route designs).
  • Gyroscope on 7 3D-scene routes (mobile only).
  • Haptics on Android (web navigator.vibrate) AND on Android + iOS in Capacitor wrapper (via @capacitor/haptics).
  • Master toggle (default OFF) + long-press settings sheet (GYRO / AUDIO / HAPTIC sub-toggles).
  • iOS web gyro permission flow (DeviceOrientationEvent.requestPermission() on toggle tap).
  • prefers-reduced-motion blocks GYRO + HAPTIC; AUDIO remains user-controllable.
  • Sonification ducks to ~0.02 gain when PRD-016 narration is playing (the two coexist).
  • Settings persist for the session only via in-memory state (per ADR-057). NO localStorage. NO sessionStorage either — runtime-only state. Reload resets to OFF.

v1 (out of scope — deferred) ​

  • Volume slider in settings sheet (ship with tuned defaults; v1.1).
  • Headphone detection (auto-enable richer sonification on plug — v1.1).
  • Gyro for /fly porkchop magnifier (conflicts with ADR-023; defer).
  • Compass / planetarium mode (orient to true north — v1.1).
  • MIDI output for performance / education contexts (v2 candidate).
  • User-adjustable sensitivity (ship tuned constants; reopen if reviewers complain).
  • iOS web haptics (browser limitation; can't fix). iOS Capacitor haptics covered.

§must-have requirements ​

IDRequirement
M1Master sensory toggle in the nav (small waveform + compass icon, 44×44 px tap target). Default OFF. Single tap toggles all enabled sub-modalities on/off. Long-press opens settings sheet.
M2Settings sheet (bottom-sheet on mobile, dropdown on desktop) with three sub-toggles: GYRO, AUDIO, HAPTIC. All default ON when the master is enabled for the first time. State is in-memory only (lost on reload per ADR-057).
M3iOS gyro permission requested via DeviceOrientationEvent.requestPermission() on first toggle tap (never on page load). If denied, GYRO sub-toggle auto-disables; AUDIO + HAPTIC remain available. (Native Capacitor app handles permission via Info.plist declaration; no in-app permission dance.)
M4prefers-reduced-motion users see only the AUDIO sub-toggle in the settings sheet. GYRO + HAPTIC are hidden (not "disabled and visible" — hidden, per ADR-022 spirit).
M5Web Audio context is created lazily on first toggle tap (user gesture required). Suspended on visibilitychange to background; resumed on foreground. Per-route audio graph torn down + rebuilt on route change.
M6Gyroscope mapping: reference-frame calibrated (G-C) — toggle on captures the current device orientation as "home"; subsequent rotations are deltas from home. Sensitivity 0.015 rad/deg. Low-pass filter smoothed = prev × 0.85 + raw × 0.15. Dead zone ±2°.
M7Touch-drag pauses gyro (T-B) — when the user is actively touching the canvas, gyro input is suppressed. 200 ms after touch-end, gyro resumes from the new position (no snap back to home).
M8Haptics use @capacitor/haptics when running under Capacitor (PRD-015), navigator.vibrate on web. iOS web has no haptics — the HAPTIC sub-toggle is hidden when neither path is available. 12 distinct haptic patterns per RFC-020 §haptics.
M9Sonification ducks to ~0.02 gain (≈ −34 dB) when PRD-016 audio narration is playing. Restored to 1.0 gain on narration end. Both toggles remain independent — neither force-disables the other.
M10Sonification pauses entirely (gain → 0) when a screen reader is detected (navigator.userAgent heuristics + ARIA live region presence) — voice output and tonal output don't compete for the same channel.
M11Per-route sonification graphs for all 11 routes (per RFC-020 §sonification). The 5 new route designs (/science, /fleet, /iss, /tiangong, /mars) reuse the agency tone palette + regime drone instrument from the original 6 routes — no entirely new instrument families.
M12Bundle size budget: < 15 KB minified gzipped for the entire sensory layer (audio.ts + haptics.ts + sensor.ts + sensory.ts). Web Audio API is native; only oscillator setup + gain graph code adds to bundle.
M13Performance budget: no 60 fps drop on the lowest-tier supported device (mid-range Android per PRD-015). Sonification CPU < 0.3 % steady-state.
M14Capacitor build adds zero MB to the mobile bundle for the sensory layer itself (Web Audio + Capacitor Haptics are runtime APIs). The 15 KB JS lives in the existing app shell.

§should-have requirements ​

IDRequirement
S1Recalibrate gesture: triple-tap on the screen recaptures gyro home orientation. Visual flash confirmation (~150 ms teal).
S2First-time toggle-on shows a non-modal toast: "Tilt to orbit · sound on · vibration on". Dismisses on next tap or after 4 s. Shown once per session.
S3Sub-toggle state in the settings sheet animates between on/off (≤ 150 ms).
S4/iss and /tiangong selected-module tones get a short envelope on selection (attack ≤ 50 ms) so the change is perceived as an event, not a glitch.
S5When AUDIO is on but no other modality, the master toggle icon shows a subtle pulse during active sonification (visual cue that audio is playing).

§will-not-have (v1) ​

  • Persistent settings across reloads (in-memory only per ADR-057; revisit in v1.x only if data justifies a single-cookie bitset).
  • iOS web haptics (no browser support; can't fix).
  • Tone.js or any external audio framework (Web Audio raw API only).
  • /fly porkchop magnifier gyro control (conflicts with ADR-023).
  • User-adjustable sonification sensitivity / volume slider (v1.1).
  • Compass / planetarium-mode camera reference (v1.1).
  • MIDI output (v2).
  • Per-user listening profile / "save my mix" (out of scope; runtime-only).

§success-criteria ​

Editorial:

  1. A first-time visitor toggles sensory on, walks through /explore, /fly (one mission arc), and /moon, and the audio difference between the three routes is immediately perceptible (verified by 3 reviewer listens before ship).
  2. The Kepler chord on /explore lands as recognisable harmony (not noise) on the device's built-in speaker without headphones — verified on iPhone SE 3 + Pixel 5.

Technical: 3. v1 ship adds < 15 KB minified gzipped to the bundle. 4. No 60 fps drop on a Pixel 5 / iPhone SE 3 with sensory enabled across all 11 routes. 5. Audio coexistence with PRD-016 narration verified: turning narration on while sonification plays drops sonification to ~0.02 gain within 50 ms; restored within 200 ms after narration ends. 6. iOS web gyro permission flow shows the system prompt only on toggle tap (not on page load); denial gracefully disables GYRO sub-toggle without affecting AUDIO/HAPTIC.

Operational: 7. Manual test matrix passes on: iPhone SE 3 (iOS Safari), iPhone 14 Pro (Capacitor wrapper), Pixel 5 (Chrome), Pixel 5 (Capacitor wrapper), Galaxy A-series (Chrome), MacBook + Chrome (desktop AUDIO only), MacBook + Firefox (desktop AUDIO only).

§dependencies ​

  • PRD-015 / RFC-018 (mobile wrapper) must ship first — the haptics integration uses @capacitor/haptics from the wrapper's plugin list.
  • PRD-016 / RFC-019 (audio narration) must ship first — the sonification ducking rule (M9) requires the narration system to exist.
  • ADR-057 (no localStorage) — constrains settings persistence to in-memory only.
  • ADR-022 (reduced motion fallback path) — constrains the GYRO + HAPTIC visibility under reduced-motion preference.
  • ADR-025 (accessibility — tier 1 claims) — constrains the screen-reader interaction (M10).
  • ADR-018 (44 px touch targets) — constrains all sensory UI elements.
  • ADR-023 (mobile magnifier) — blocks /fly porkchop gyro control.

§resolved decisions ​

Resolved 2026-05-16 in conversation with Marko.

  1. Sonification scope — RESOLVED: All 11 routes (not just original 6 prototypes). Adds 5 new per-route sonification designs in RFC-020 §sonification: /missions, /science, /fleet, /iss, /tiangong, /mars.
  2. Capacitor haptics — RESOLVED: @capacitor/haptics in Capacitor builds, navigator.vibrate fallback on web. iOS gets basic Taptic Engine feedback when wrapped (not on iOS web).
  3. Audio narration coexistence — RESOLVED: Sonification ducks to ~0.02 gain while narration plays. Both toggles remain independent. Restored 200 ms after narration ends.
  4. Settings UI — RESOLVED: Two separate buttons in the nav (narration speaker icon + sensory waveform icon). Each has its own bottom-sheet. One icon = one capability.
  5. Storage — RESOLVED: in-memory only. ADR-057 forbids localStorage; the original RFC's "v0.4 reconsider localStorage" line is deleted entirely. Reload resets the master toggle to OFF.
  6. Gyro mapping (G-C), gyro+touch (T-B), per-screen sonification (S-C), haptics+audio interaction (I-B), toggle UX (U-2), iOS permission timing (P-A) — RESOLVED: all 6 original RFC choices stand. Mature design; no rework needed.

Additional decisions resolved 2026-05-16 (second batch):

  1. Default master-toggle state — RESOLVED: Default OFF, with a one-time landing-page hint ("tilt + sound + buzz — try sensory mode") dismissable toast on /. Opt-in is more polite + avoids the "what's making noise" reaction on shared / public devices.
  2. prefers-reduced-motion × AUDIO — RESOLVED: AUDIO remains user-controllable under reduced-motion. Only GYRO + HAPTIC are hidden. Reduced-motion is a motion preference, not an audio preference.
  3. Recalibrate gesture — RESOLVED: Triple-tap on canvas. Discoverable via help text in settings sheet. 150 ms teal flash confirms.

§remaining follow-ups ​

  1. Sonification authoring review for the 5 new routes. RFC-020 §3.3 proposes designs based on each route's character; need a listening-test pass on each before ship. Marko + 1 reviewer per design = 5 listening sessions. Operational, not architectural.
  2. iPhone SE 3 audio quality test. The Kepler chord on built-in speakers (not headphones) at the SE's small driver — does it sound musical or muddy? Test before ship; tune oscillator gains down if muddy. Operational.
  3. Triple-tap × iOS accessibility shortcut conflict. iOS allows triple-tap to be bound to system accessibility features. If a real user collision surfaces during testing, switch the recalibrate gesture to a dedicated settings-sheet button. Implementation-time decision.

PRD-017 · Orrery · Sensory Layer · Drafted 2026-05-16 · Closes-into-RFC-020

Orrery — architecture documentation · MIT · No tracking

© Marko Dragoljević and contributors