ADR-074 — Panorama schema v2 + spatial-context renderer; commit to "exhibit, not embed"

Status · Accepted Date · 2026-05-31 Closes into · PRD-022 (product framing), #286 (issue) Related ADRs · ADR-061 (region_bounds on SurfaceSite — sibling schema-extension pattern), ADR-062 (sphere → flat-patch transition — wraps the panorama entry point), ADR-072 (shared SurfaceScene.svelte — owns the panorama enter/exit handoff), ADR-001 (Three.js r128 pin — constrains sprite + skybox API) Related issues · #283 (Surface Hotspots v2 — hard prereq, shipped), #285 (Earth launchpads — adjacent surface routes), #286 (this), #287 (/explore 4K planet textures — orthogonal layer)

Context

PRD-022 frames the user-facing problem and commits to the "spatial-context, not raw-quality competition" design lens for the Tier-3 panorama experience. This ADR locks the technical shape:

• The schema extensions on the surface-hotspots.json entry
• The renderer changes inside src/lib/hotspot-tier3-skybox.ts
• The HUD component decomposition (caption / compass / annotation card / cycler / cross-link)
• The annotation-sprite math (yaw/pitch → 3D position on the inverted-sphere interior)
• The honest-sky / honest-nadir render strategy
• Migration path for the 28 existing panorama entries

Decision space

Schema: extend existing field or add sidecar?

Today the panorama is a single string URL on the hotspots entry:

{ "hotspot_tier3_panorama": "/images/hotspots/mars/perseverance/tier3-pan.jpg" }

Three options:

• (A) Type-widen the field: hotspot_tier3_panorama: string | PanoramaObject. Caller branches on typeof. Backwards-compatible for existing entries. Loose typing; UI code has two paths.
• (B) Sidecar fields: keep hotspot_tier3_panorama as the default-URL fallback, add new top-level fields panorama_metadata, panorama_annotations, panorama_set on the same entry. Caller reads new fields when present, falls back to the legacy URL.
• (C) Replacement field: panorama: { default: { url, metadata, annotations }, set: [...] }. Cleanest typing. Requires migration of all 28 existing entries before any code can ship.

Renderer: extend hotspot-tier3-skybox.ts or add a parallel renderer?

Today's renderer is 221 LOC, single-purpose. Three options:

• (A) Extend in place: add caption/compass/annotation handles to the existing SkyboxHandle interface; route attaches HUD components and reads handle state. Single source of truth for skybox internals.
• (B) Wrap in a higher-level "PanoramaExhibit" component: new Svelte component owns the skybox handle plus the HUD; route uses the wrapper. Cleaner separation; one more abstraction layer.
• (C) Compose by composition: keep createSkybox minimal; add separate factories for annotation sprites, compass yaw readout, caption metadata loader. Caller composes. Most flexible; most pieces to track.

Annotation sprites: 3D sprites inside sphere or 2D HUD overlay?

Annotations need to "stick to" a yaw/pitch direction so when the user drags-orbits the camera, the annotation tracks with the terrain feature behind it.

• (A) 3D THREE.Sprite inside the sphere: positioned at radius ≈ 80 in the yaw/pitch direction. Camera always-faces, scale-invariant. Click via raycaster. Native Three.js, r128-supported.
• (B) 2D HTML overlay with manual yaw/pitch → screen-space projection: per-frame transform of yaw/pitch into screen top/left. CSS sprite. Click via DOM event.
• (C) THREE.Object3D group with billboarded plane geometry: similar to (A) but more flexible per-annotation geometry. Heavier.

Free pitch: drop clamp entirely or extend with microcopy zones?

Today the pitch is clamped to ±20°. Removing the clamp means the user can look at synthetic-fill regions.

• (A) Hard ±60° clamp: looser than today but still hides the worst of the synthetic regions.
• (B) Full ±90° + microcopy in synthetic regions: drop clamp; when camera enters a synthetic-region pitch range (declared in panorama_metadata.synthetic_regions), overlay text "This region of the sky was not photographed — pattern is synthetic." Honest.
• (C) Per-site clamp variable: some sites cap at ±60° (panorama doesn't even cover ±90° in source), some at ±90°. Tunable.

Honest-sky asset treatment

Current Mars panoramas have a procedural orange-gradient fill above the photographed horizon. Three options:

• (A) Re-source with real Mars sky: NASA Mastcam-Z panoramas often have real sky pixels above the horizon. Re-sourcing for Curiosity / Perseverance likely yields real-sky data.
• (B) Alpha-out the synthetic fill + render synthetic-pattern fill (light-dotted graticule) in its place: turn the panorama JPEG into RGBA, alpha = 0 where pixels are procedural; the skybox renders the alpha-cleared region as a synthetic pattern controlled by the renderer (CSS-like grid).
• (C) Leave the procedural fill but caption it as synthetic: cheapest; relies on the microcopy + caption to disambiguate. User still sees a gradient that looks like sky.

Decisions

Schema — Option B (sidecar fields)

Keep hotspot_tier3_panorama as the legacy URL. Add new optional fields on the same hotspots entry:

{
  "id": "perseverance",
  "hotspot_tier3_panorama": "/images/hotspots/mars/perseverance/tier3-pan.jpg",
  "panorama_metadata": {
    "sol": 46,
    "date": "2021-04-06",
    "instrument": "Mastcam-Z",
    "caption": "Jezero Crater floor with sample tubes deposited at Three Forks depot. Ingenuity helicopter visible right.",
    "credit_team": "NASA/JPL-Caltech/ASU",
    "nasa_id": "PIA24906",
    "real_extent_pct_vertical": 35,
    "synthetic_regions": [
      { "pitch_min_deg": 15, "pitch_max_deg": 90, "kind": "synthetic_sky" }
    ]
  },
  "panorama_annotations": [
    {
      "id": "ingenuity",
      "yaw_deg": 80,
      "pitch_deg": -5,
      "label": "Ingenuity helicopter",
      "body": "First powered flight on another planet — Apr 19, 2021"
    }
  ],
  "panorama_set": [
    { "id": "sol-46", "url": ".../tier3-pan-sol46.jpg", "metadata": {...}, "annotations": [...], "default": true },
    { "id": "sol-1023", "url": ".../tier3-pan-sol1023.jpg", "metadata": {...}, "annotations": [...] }
  ],
  "traverse_stop_link": "stop-3"
}

Why B: Lets us land schema + Phase 2 renderer + curate metadata + curate annotations in independent slices. No upfront migration of all 28 entries before any code ships. Sites without panorama_metadata fall back to a generic caption ("Surface panorama — agency / source"). The eventual end state (all marquee sites have full sidecar fields, remaining 21 have legacy URL + generic caption) is honest about what's curated.

Resolution rule (deterministic, easy to reason about):

1. If panorama_set is present and non-empty, the default panorama is the entry with default: true (fall back to first entry). The cycler UI shows when panorama_set.length > 1.
2. Else if hotspot_tier3_panorama URL is set, that's the only panorama. Cycler hidden.
3. panorama_metadata + panorama_annotations at the entry root are the metadata/annotations for the default panorama (the legacy single-pano case).
4. Inside panorama_set each entry carries its own metadata + annotations — those override the entry-root values for that set member.

Renderer — Option A (extend in place) + Option B (HUD components)

Hybrid:

• Extend hotspot-tier3-skybox.ts with new handle methods for: annotation sprite mount/unmount, yaw readout subscription, synthetic-region pitch check. The skybox owns Three.js objects.
• Add a new Svelte component layer for the HUD pieces: PanoramaCaptionOverlay, PanoramaCompassRose, PanoramaAnnotationCard, PanoramaCycler, PanoramaCrossLink. The route assembles them and passes data + skybox handle.

The route stays the integration point (no new "Exhibit" wrapper). HUD components are small, single-purpose, and easy to test in isolation.

Why A + B: Keeps the renderer's Three.js logic in one file. HUD pieces are Svelte components like the rest of the route's HUD (ViewToggleButton, LayerChipRow). No new abstraction layer; same pattern as the existing surface-scene HUD.

Annotation sprites — Option A (3D THREE.Sprite inside the inverted sphere)

For each annotation { yaw_deg, pitch_deg, label, body }:

const yaw = THREE.MathUtils.degToRad(yaw_deg);
const pitch = THREE.MathUtils.degToRad(pitch_deg);
const r = SKYBOX_RADIUS - 1;  // sit just inside the sphere
const x = r * Math.cos(pitch) * Math.sin(yaw);
const y = r * Math.sin(pitch);
const z = r * Math.cos(pitch) * Math.cos(yaw);

const spriteMaterial = new THREE.SpriteMaterial({ map: pinTexture, depthTest: false, transparent: true });
const sprite = new THREE.Sprite(spriteMaterial);
sprite.position.set(x, y, z);
sprite.scale.set(2, 2, 1);
sprite.userData = { annotationId, label, body };
group.add(sprite);

Click handling via Raycaster.intersectObjects([sprite]) on canvas click; if hit, emit a Svelte event that opens the PanoramaAnnotationCard.svelte with the annotation's label + body. Sprite scale is constant in world units; appears smaller at the skybox edge (perspective correct).

Why A: r128-native, no per-frame projection math, raycaster integrates naturally with the existing canvas-click flow used by tier-2 hotspot markers, screen-readers can be served via aria-live overlay parallel announcements (same pattern as the existing PanoramaOverlay.svelte sr-only span).

Sprite texture: small (32 × 32 px) pin-shape with an outline, agency-tinted from the site's agency colour. Bundled in static/images/ui/panorama-annotation-pin.png (single asset, tinted at runtime via spriteMaterial.color).

Free pitch — Option B (full ±90° + microcopy in synthetic regions)

Drop the ±20° clamp entirely. Allow full ±90° drag. When the camera's current pitch is inside one of the panorama_metadata.synthetic_regions pitch ranges, render a PanoramaSyntheticRegionMicrocopy.svelte overlay centred on the canvas:

"This region of the sky was not photographed at this site — the visible pattern is synthetic fill."

Microcopy text is i18n'd (paraglide). Microcopy hides when the camera leaves the synthetic-region pitch range.

For sites without synthetic_regions declared (legacy single-URL panoramas), no microcopy fires — but free pitch still works.

Why B: Honest. The current ±20° clamp is hiding-the-problem, not solving it. Letting the user look up and see "no data here" is better than letting them look up and see fake sky.

Honest-sky asset treatment — Hybrid A + B

Per-site decision (recorded in panorama_metadata.synthetic_regions):

• Curiosity, Perseverance: try Option A first (re-source with real Mars sky data from NASA Mastcam-Z 8K panoramas where available). If re-source succeeds, synthetic_regions = [] — no microcopy. If re-source fails or only 4K is available, fall back to Option B (alpha-out the procedural region + synthetic-pattern render).
• Apollo 11, Apollo 17: keep the honest black sky as-is. synthetic_regions = [] (or just the grey-fill nadir block, depending on what we do there).
• Chang'e 4, Chandrayaan-3, Mars 3: re-source if higher-res available; otherwise alpha-out procedural fills and declare synthetic regions. Mars 3 specifically: the original is so brief (~14.5 s of transmitted data) that the "panorama" is mostly synthetic — declare honestly and lean into it as a historical artifact.

Phase 1 includes a per-site decision table to be filled in during the re-source pass.

Why hybrid: Per-site honesty beats one-size-fits-all. Some sources are good enough to use raw; others need transparent honest-fill.

Compass — yaw readout on SkyboxHandle

Add getYaw(): number and setYaw(yawDeg: number): void to SkyboxHandle. Surface route's PanoramaCompassRose.svelte subscribes via per-frame poll inside the existing animation loop (no new RAF). Compass rose rotates with negative yaw so the N-arrow points to the panorama's "north" (the panorama's 0° yaw direction, by convention pointing at the rover's forward direction OR the lander's "up" direction — declared in panorama_metadata.compass_zero_direction).

For sites without compass_zero_direction declared, the N-arrow is hidden (no false orientation claim) — only the cardinal "you're looking ↺ this way from start" arrow shows.

Cycler + Cross-link — straight Svelte components

PanoramaCycler.svelte renders left/right arrows when panorama_set.length > 1. Click → calls skybox handle's swapTexture(url) method (new, added in Phase 2A) + updates the parent's currentPanoramaId state. Caption + annotations re-render from the new panorama's data.

PanoramaCrossLink.svelte renders a slim caption-bar bottom-right with 1-3 of: traverse-stop link, fleet-entry link, audio-episode link. Each is <a> tag with route href (/mars?site=perseverance&traverse_stop=stop-3, /fleet/perseverance, /?audio=perseverance-sol-46). Visible only when at least one link target exists.

Migration of existing 28 entries

Phase 0 doesn't migrate. Phase 4 (curation) authors:

• 7 marquee sites: full sidecar fields (panorama_metadata + panorama_annotations).
• Remaining 21: minimal panorama_metadata (just caption + credit_team from existing image-provenance entries; no sol/date/annotations). Falls back gracefully — caption + compass work everywhere; annotation card never opens because there are no annotations; cycler stays hidden.

Schema enforces neither — panorama_metadata and panorama_annotations are optional. The "ship the schema first, curate over time" pattern.

Render-loop cost

• Annotation sprite raycast: existing raycaster is fired once per click event (not per-frame). No additional per-frame cost.
• Compass yaw poll: one Object3D getWorldDirection + atan2 per frame. Negligible.
• Synthetic-region pitch check: one comparison per frame against the active panorama's synthetic_regions array (≤3 entries typical). Negligible.

No regression in 60 fps target on /moon or /mars desktop or mobile. Verified in Phase 2 e2e + manual recording.

Bundle + asset budget

• New Svelte HUD components (5 small files, ~50-100 LOC each) — negligible bundle impact (~5 KB minified).
• Annotation pin sprite (32×32 PNG, ~1 KB) — single asset, tinted at runtime.
• Per-panorama metadata + annotations — embedded in surface-hotspots.json (~200 bytes per site for marquee 7 = ~1.4 KB total). Loaded with the existing surface-hotspots fetch.
• Re-sourced higher-res panoramas — per-site decision; budget per Phase 1 design. 8K source for Curiosity + Perseverance only ≈ ~10 MB total (5 MB each at q80) — under workbox cache cap (~2× the current 4K asset for those two sites).

Migration + validate-data

static/data/schemas/surface-hotspots.schema.json extended with:

{
  "definitions": {
    "PanoramaMetadata": { "type": "object", "properties": { ... } },
    "PanoramaAnnotation": { "type": "object", "required": ["id", "yaw_deg", "pitch_deg", "label"], ... },
    "PanoramaSetEntry": { "type": "object", "required": ["id", "url"], ... }
  },
  "properties": {
    "panorama_metadata": { "$ref": "#/definitions/PanoramaMetadata" },
    "panorama_annotations": { "type": "array", "items": { "$ref": "#/definitions/PanoramaAnnotation" } },
    "panorama_set": { "type": "array", "items": { "$ref": "#/definitions/PanoramaSetEntry" } },
    "traverse_stop_link": { "type": "string" }
  }
}

validate-data extended to:

• Cross-check traverse_stop_link resolves to a stop in the corresponding mars-traverses/<mission>.json or moon-traverses/<mission>.json (where the file exists).
• Cross-check panorama_set[].url files exist on disk (mirror existing hotspot_tier3_panorama check).
• Validate panorama_metadata.synthetic_regions pitch ranges are within ±90°.

i18n

Caption + annotation strings authored in en-US in surface-hotspots.json. Paraglide overlay pattern (same as moon-sites / mars-sites) puts per-locale strings in static/data/i18n/<locale>/panorama-strings/<site-id>.json. The loader reads the localised override when the user's locale isn't en-US; falls back to the en-US strings in the canonical hotspots file.

Microcopy ("This region of the sky was not photographed…", compass N-arrow tooltip, cycler arrow labels, cross-link labels) lives in messages.json (paraglide message keys), localised by the standard paraglide pipeline.

A11y

• Caption overlay has role="region" + aria-label="Panorama caption" — screen readers announce on entry.
• Annotation sprites are duplicated as <button> elements in an sr-only list inside PanoramaOverlay.svelte — keyboard-navigable, focusable, click triggers the same caption-card open.
• Compass rose has aria-label="Compass — currently facing <direction>" updated as yaw changes.
• Cross-link bar is plain <a> elements — natural focus order.
• Reduced-motion: caption fade-in skipped; compass rose updates without easing; auto-tour (Phase 3) replaced with manual prev/next list.

Open questions / forks left for Marko

1. Marquee 7 — swap Apollo 17 for SLIM (JAXA)? Better 3-3 NASA/non-NASA balance; cost is dropping one of the apex Apollo missions from marquee status. Either path defensible.
2. Mars 3 specifically: the source is so degraded (~14.5 s of transmitted scan data) that the "panorama" is mostly synthetic. Two readings:
   • (a) Include as marquee + lean into the historical-artifact framing in metadata + caption ("First Mars soft landing — 14.5 s of transmitted data before signal lost; the panorama is reconstructed from this fragment"). Honest.
   • (b) Demote to non-marquee; replace marquee slot with another Mars mission (Spirit, Opportunity — both NASA). Default in this ADR: option (a). Calling it out for review.
3. Cycler UX on mobile: tiny arrows or swipe gesture? Mockup mocks arrows; gesture deferred to Phase 3 polish if signal warrants.
4. Auto-tour scope (Phase 3C): replicate a NASA-style "Play tour" that pans + reads annotation captions, or simpler "next annotation" stepper? Default: simpler stepper; explicit auto-pan deferred.

Anti-decisions — explicitly ruled out

• No vision-API annotation generation. Curation is editorial; Claude / GPT cannot reliably identify "Vera Rubin Ridge" vs "an unnamed ridge" without human verification, and hallucinated annotations would corrode the "honest exhibit" framing.
• No VR / WebXR mode. Future possibility; out of scope for this initiative.
• No tile-pyramid streaming. PRD-022 explicitly defers — 8K base for marquee 7 is the asset ceiling.
• No replacement of the inverted-sphere skybox primitive. Cube-map / panoramic-cylinder / 360-video are all alternatives; staying with the existing primitive simplifies the migration.
• No camera-autopilot to annotations. Click-to-open caption card, then user-driven pan. Auto-tour (Phase 3C) is the explicit opt-in.

References

• Existing renderer: src/lib/hotspot-tier3-skybox.ts (221 LOC)
• Existing schema: static/data/schemas/surface-hotspots.schema.json (no panorama fields beyond hotspot_tier3_panorama: string today)
• Surface-scene panorama entry: src/lib/surface-scene/SurfaceScene.svelte:2238 (PanoramaOverlay mount point)
• Sibling pattern (annotations on surface): existing hotspot_annotations: Array<{ id, label, lat_offset_m, lon_offset_m }> field — rendered at tier-2 patch level. ADR-074's annotations are panorama-space (yaw/pitch), separate concern.