ADR-069 — Three-layer overlay-completeness policy (validate-data check + opt-in smoke pre-push + AGENTS.md rule)

Status · Accepted Date · 2026-05-25 Closes into · AGENTS.md §"Data layer rules" (overlay-completeness rule + history note) Related ADRs · ADR-046 (image-provenance validator), ADR-051 (link-provenance validator), ADR-052 (symmetric fleet ↔ mission link validator) Triggered by · GH #83 e2e regression (May 2026)

Context

GH #83 ("unpack better satellite orbits on Earth and support in science") shipped on 2026-05-24 across two commits — 27eec78e7 (en-US infra + 11 new constellation entries in static/data/earth-objects.json) and e84ada626 (13-locale translations). Local preflight passed, pre-push hook passed, CI on the head commit reported green for CI and docker-e2e's build stage. Then e2e (Playwright suite, runs on push-to-main) turned red with eleven runtime SvelteKit 404s:

[WebServer] SvelteKitError: Not found: /data/i18n/en-US/earth-objects/starlink.json
[WebServer] SvelteKitError: Not found: /data/i18n/en-US/earth-objects/oneweb.json
... (9 more)

The defect: when adding a new ID to earth-objects.json, the contributor must also create the matching i18n overlay at static/data/i18n/en-US/earth-objects/<id>.json. The base file declares the record; the overlay supplies the localised name / short / description / scale_fact. Without the overlay, /earth route hydration 404s on first paint when the user views the marker detail panel. Eleven such overlays were missing because the work focused on the base registry, the schema, and the translation pipeline — none of which require the en-US overlay to be present.

This same defect class affects five other registries that follow the same base + overlay shape: static/data/fleet/index.json, static/data/moon-sites.json, static/data/mars-sites.json, static/data/science/<tab>/_index.json (both base file and overlay), and the multi-file iss-modules / tiangong-modules / iss-visitors / tiangong-visitors overlays. An audit at the time of this ADR found 18 orphan IDs across the project — 15 in the just-shipped #83 work and 3 in concurrent parallel-agent work (luna16 / luna21 / beresheet, added to moon-sites.json by the moon-tier3 batch but never given overlays).

The existing safety nets:

• typecheck type-checks TypeScript + Svelte source but never reads runtime JSON data.
• lint style-checks code but never walks static/data/.
• test unit-tests individual modules but no test enumerates the per-registry overlay coverage.
• validate-data runs AJV schemas + provenance + symmetric-link checks but had no overlay-completeness check — its mental model was "is each individual file well-formed", not "does the cross-reference graph between registries close".
• build (SvelteKit adapter-static) compiles the route output but the missing overlays only 404 at runtime when the /earth panel actually requests them, not at build time.
• e2e (Playwright) catches the regression at the right moment but runs only on CI push-to-main, not pre-push.

The asymmetry: the preflight chain that the pre-push hook runs costs ~3 minutes on a developer laptop, the e2e chain costs ~16 minutes and is gated to CI. Most of the time this is the right trade. For the specific defect class of "missing overlay file" it's the wrong one — the regression is determinate and cheap to detect locally if you look for it specifically, but expensive to detect by running every Playwright spec.

The decision: close the specific gap with cheap, targeted local-only enforcement, layered so that the catching net is graded by certainty + cost.

Decision

A three-layer policy enforces overlay-completeness at increasing scope + cost, layered from cheapest-always-on to most-thorough-opt-in.

Layer 1 — validate-data overlay-completeness check (always-on)

scripts/validate-data.ts gains a new validation stage that runs after image-pipeline-manifest validation and before process-exit. For each of the registries below it iterates all declared IDs and verifies that the corresponding en-US overlay file exists on disk; missing overlays accumulate into the same overlayFailed counter that participates in the final non-zero-exit gate.

Base registry	Overlay path checked
static/data/earth-objects.json (top-level array, id field)	static/data/i18n/en-US/earth-objects/<id>.json
static/data/fleet/index.json (top-level array, id + category fields)	static/data/i18n/en-US/fleet/<category>/<id>.json
static/data/moon-sites.json (top-level array, id field)	static/data/i18n/en-US/moon-sites/<id>.json
static/data/mars-sites.json (top-level array, id field)	static/data/i18n/en-US/mars-sites/<id>.json
static/data/science/<tab>/_index.json (ids array)	both static/data/science/<tab>/<id>.json (base) AND static/data/i18n/en-US/science/<tab>/<id>.json (overlay)

Cost on a fresh node tree: ~50 ms total. The check runs inside the existing validate-data invocation, so it participates automatically in the preflight chain (typecheck → lint → test → validate-data → check-no-secrets → build) and therefore in the pre-push hook. No new dependency, no new wiring. Other locales are not checked — the runtime i18n loader falls back to en-US for any missing locale-specific overlay, so en-US presence is the necessary + sufficient guarantee that a runtime 404 won't occur.

Layer 2 — opt-in smoke-subset e2e in pre-push (off by default)

A new npm script test:e2e:smoke runs only tests/e2e/landing.spec.ts + tests/e2e/smoke.spec.ts against desktop-chromium with --workers=1. Those two specs are deliberately chosen because they together hit every top-level route once at first paint, which is precisely the moment a missing-overlay 404 surfaces. Cost: 30-60 seconds on a developer laptop after the npm run build that's already needed for any Playwright run.

.husky/pre-push gains an env-var-gated stage that runs this script after npm run preflight succeeds, only when the contributor sets PREFLIGHT_INCLUDE_SMOKE=1:

PREFLIGHT_INCLUDE_SMOKE=1 git push

Off by default because (a) daily commits don't pay the 30-60 second cost on every push, (b) Layer 1 already catches the dominant defect class, and (c) the smoke subset is only meaningful when the working tree changed something route-relevant (earth-objects, fleet data, moon-sites, mars-sites, /earth or /moon or /mars source). Recommended for release-cut pushes and for any push that touches a registry. The env-var gate is documented in .husky/pre-push comments and in AGENTS.md.

Layer 3 — AGENTS.md non-negotiable rule + registry-to-overlay table

A new paragraph under AGENTS.md §"Data layer rules" documents the registry → overlay mapping that every agent (Claude, Cursor, human, future) must respect when adding a new ID. The text is marked NON-NEGOTIABLE (matching the existing image-provenance and link-provenance rules) and includes a history note linking back to GH #83 so future agents understand why the rule exists. Layer 3 doesn't enforce anything mechanically — Layers 1 + 2 do that — but it surfaces the rule at the moment an agent is most likely to be about to break it (reading AGENTS.md before touching the data layer).

Consequences

Positive

• The exact defect class that caused GH #83 fails at preflight now, before push, on every developer machine. Layer 1 caught 18 orphan IDs the moment it activated — 15 from the just-shipped #83 work, 3 from concurrent parallel-agent work — and forced them to be backfilled in the same commit that introduced Layer 1.
• The graded design avoids the "preflight is too slow now" trap. Layer 1 adds 50 ms; daily pushes don't notice. Layer 2 stays opt-in for cases where the 30-60s is justified. The pre-push hook stays default-cheap.
• The registry-to-overlay contract is now explicit + discoverable, both in the validator code (single point of truth for the canonical paths) and in AGENTS.md (single point of agent-facing rule).
• No new dependency. Both Layers 1 + 2 use tooling already in the tree (validate-data, Playwright, husky).

Negative

• Layer 1 enforces only en-US overlay presence, not non-English locale overlays. A non-English translation that's missing won't fail validate-data; it will fall back to en-US at runtime. This is the right trade — locale completeness is owned by a separate i18n pipeline, and tying validate-data to it would slow daily preflight unnecessarily — but it means an agent could in principle ship a feature with English-only text where the project expects 14-locale coverage. Mitigation: the translation pipeline has its own audit step that catches this asymmetrically.
• Layer 2 requires the contributor to remember to set the env var. A push that changes earth-objects.json but doesn't set PREFLIGHT_INCLUDE_SMOKE=1 could still in principle reach origin with a runtime 404 that Layer 1 missed. The mitigation is that Layer 1 already catches the specific overlay-missing class; Layer 2 catches secondary issues (route handler bugs, broken navigation links, etc.) which are a different defect family. In practice the env var is a release-readiness aid, not a daily-push gate.
• 18 orphan overlays were scaffolded in the activation commit using generated descriptions. The fleet/constellation overlays use authored descriptions; the moon-site overlays (luna16 / luna21 / beresheet) use hand-written historical fact entries. Quality is "ship-ready" but not "translator-polished" — the i18n pipeline can refine them later.

Neutral

• validate-data runtime grows from ~3 s to ~3.05 s. Negligible in the preflight chain.
• .husky/pre-push is now 35 lines instead of 25. Still a single bash script with a single env-var branch.

Audit findings folded in

At activation time, Layer 1 surfaced 18 orphan IDs across the project. All 18 overlay files are added in the same commit as the validator change so the activation push is clean:

Registry	IDs
fleet/index.json (category constellation)	starlink, oneweb, iridium-next, kuiper, planet-labs, sentinel-copernicus, landsat, gps-gnss, galileo-gnss, glonass, beidou, goes-noaa, inmarsat, tundra-molniya, o3b-mpower (all 15 from GH #83)
moon-sites.json	luna16, luna21, beresheet (3 from concurrent parallel-agent moon-tier3 work)

Trail

• GH #83 root-cause regression: 2026-05-24, fix in fe8106e50 (overlay backfill only — did not prevent recurrence)
• Three-layer policy: commit 4df909dac 2026-05-25 — adds Layers 1, 2, 3 + backfills the 18 audit-found orphans
• AGENTS.md "overlay-completeness rule" with full registry table: same commit
• Pre-push hook env-var gate: .husky/pre-push same commit

Future extensions

• If a sixth registry following the base + overlay shape lands (likely candidates: a new cislunar-objects.json for the v0.8 NRHO/DRO work, a tianlu-modules.json for a hypothetical Chinese cislunar station), add it to Layer 1's check list. Cost per new registry: ~10 lines of validator code.
• If the project later adds a stricter "every locale must have every overlay" expectation (e.g. a translation-completeness gate before a release tag), extend Layer 1 from "en-US only" to "all 14 locales" but keep that stricter check behind a release-only env var so daily preflight cost doesn't grow.
• If Layer 2 turns out to be set by every contributor anyway (suggesting the cost is acceptable as a default), flip the default and let PREFLIGHT_SKIP_SMOKE=1 be the opt-out instead.