RFC-028 — data.ts split + DAL architecture review (v0.7)

Status · Draft Date · 2026-06-13 Target · v0.7 Tracked by · GH issue #327

Why this is an RFC. src/lib/data.ts is 1,565 lines, 83 exports, 37 caller files — one module owning every JSON fetch the app does, every locale overlay, every gallery walker, plus four module-level singletons for provenance + audio + episode-source manifests. The issue tracker frames this as a split, but the real ask is an architecture review: cache shape, locale-overlay handling, where module state belongs, and whether provenance is a DAL concern at all. This RFC locks the answers to those questions BEFORE Phase 1 (the mechanical split) starts, so the slices don't have to re-litigate design choices mid-flight.

Gating sentence: Locks the post-split DAL architecture — shared cache behind a getCached<T>() accessor, a single withLocaleOverlay<T> helper, lazy-singleton domain state with explicit per-domain reset hooks, provenance moved out of the DAL into $lib/provenance/, a new galleries-data.ts extraction that breaks the only cross-domain edges in the current call graph, and a 10-slice Phase 1 sequence keeping the public-API barrel — tracked by issue #327.

Goal

End up with ~9 domain modules under $lib/ (plus $lib/provenance/) that each own a single subject area, a data-core.ts that owns the cache + the locale-overlay helper, and a thin data.ts barrel that re-exports the public API so the 37 existing callers don't change. Lock the architectural choices now so the file split itself is mechanical.

Scope (v0.7)

Phase 0 (this RFC) — answer the 7 architectural questions from #327, plus the cross-domain gallery question surfaced during the assessment.
Phase 1 — the mechanical split itself: ~10 slices, one per domain module (+ one for data-core, one for the provenance move). Each slice independently revertable + browser-verified.
Caller sweep (drop the barrel + migrate the 37 from '$lib/data' imports to per-domain modules) is out of scope for this RFC. File a separate issue if bundle measurements show tree-shaking actually moves the needle once the split lands.

Non-goals (v0.7)

TanStack-Query / query-key migration — a real architecture change with SSR + hydration implications. Document the migration path here, but defer to v0.9 alongside the i18n caption pipeline (#257).
Cache TTL / LRU bound — at ~100 distinct URLs cached the leak is theoretical. Document the static-site assumption; revisit only if we ever go SSR or per-user.
Per-domain caches — accessor pattern (see Q1 answer) keeps the cache shared without exposing the Map. Per-domain caches are an unnecessary indirection at this corpus size.
Schema/type consolidation — $types/* already owns shape definitions; this RFC only moves runtime code.
Caller sweep — Phase 1 ships a thin barrel; sweep is a separate issue, gated on a real tree-shaking win.

Survey — current `data.ts` shape (verified 2026-06-13)

1,565 lines, 83 exports, 37 caller files (matches #327's audit).
Layers:
- Core: one cache = new Map<string, unknown>() + async function get<T>(path, fetchFn). 25 LOC.
- Domain getters: 83 exports across 8+ subject areas. ~1,480 LOC.
- Module state: 3 lazy singletons (provenanceManifest, audioProvenance, episodeSources) + 1 derived index (provenanceIndex).
Inter-getter call graph (audited):
- Intra-domain calls are common and acyclic: getMissionsForLibrary → getMission, getPlanets → planets(), etc.
- Cross-domain calls all live in gallery code:
  - getMoonSiteGallery → getMissionGallery + getFleetGallery
  - getMarsSiteGallery → getMissionGallery + getFleetGallery
- 4 of 4 cross-domain edges are gallery → gallery fan-out to the right "source" domain. Removing galleries from the per-domain modules makes the rest of the graph a clean DAG.
Internal helpers (not exported): surfaceHotspotsSidecar(), hotspotMetadataOverlay(), getCategoryGallery(). Used only by surface-site + gallery getters.
Latent bug worth flagging: __resetCache() clears the URL cache + provenanceManifest + provenanceIndex but not audioProvenance or episodeSources. Tests that rely on cold-state audio/episode-source loading would silently get stale state if run after another test exercised those getters. Phase 1's per-domain reset hook design fixes this for free.

The 7 architectural questions — locked answers

Q1 · Per-domain caches vs shared cache singleton?

Answer: shared cache, behind a getCached<T>() accessor. Don't export the Map.

data-core.ts exports:

export async function getCached<T>(path: string, fetchFn: FetchLike = fetch): Promise<T>;
export function clearAllCaches(): void; // umbrella — calls per-domain reset hooks too

The internal Map<string, unknown> is module-private. Domain modules can't accidentally re-declare it (foot-gun Q7) because they don't have access to it. Invalidation policy stays in one place. URL-keyed cache is naturally global — at our static-site scale ~100 distinct URLs, per-domain partitioning would add complexity without any measurable benefit.

Q2 · Lazy module singletons or query-key pattern?

Answer: keep lazy singletons for v0.7. Document the migration path to a TanStack-Query-style pattern as a v0.9 follow-up.

Rationale: switching the 4 manifest-style getters to a query-key pattern is a real architecture change with SSR/hydration implications, and +page.server.ts-shaped data flow we don't have today. Out of scope for "split the file." But the split itself must preserve the lazy-singleton pattern cleanly + with proper reset coverage (see Q5 + the latent-bug note above).

Q3 · Locale-overlay merge — shared transformer or inline?

Answer: shared. New helper in data-core.ts:

export interface WithLocaleOverlayOptions<TBase, TOverlay, TOut> {
  basePath: string;
  overlayPath: string;
  locale: string;
  merge: (base: TBase, overlay: TOverlay | null) => TOut;
}

export async function withLocaleOverlay<TBase, TOverlay, TOut>(
  opts: WithLocaleOverlayOptions<TBase, TOverlay, TOut>,
): Promise<TOut>;

Audit shows 7+ getters re-implementing the same fetch base → fetch overlay (catch + ignore 404) → merge pattern (missions, planets, fleet, science, sun, ISS, Tiangong). Three different merge styles exist (shallow, deep, key-based) — the explicit merge callback covers all three without forcing a single strategy.

Q4 · Barrel forever, or sweep callers?

Answer: barrel for Phase 1; sweep gated on measurement.

Phase 1 ships data.ts as a thin barrel re-exporting from the 9 domain modules — zero caller-side changes, zero blast radius. After Phase 1 lands:

Run a baseline bundle measurement on /explore (a route that today pulls in $lib/data for satellites + belts + small-body galleries but doesn't need missions/fleet/station JSON).
If dropping the barrel saves >50 KB on the route's initial JS, file a separate sweep issue.
If the saving is <50 KB, the readability win from the split is enough — keep the barrel.

Tree-shaking is the speculative benefit. Readability is the certain one. The split's value doesn't depend on the sweep.

Q5 · Provenance / source-logos / audio-provenance — DAL or separate concern?

Answer: separate concern. Move out of the DAL entirely.

Create $lib/provenance/ (not provenance-data.ts):

$lib/provenance/
  image-manifest.ts      — getImageProvenanceManifest + getImageProvenance + the singleton + reset hook
  source-logos.ts         — getSourceLogos + getTextSources + (tiny) singleton + reset
  audio-manifest.ts       — getAudioProvenanceManifest + singleton + reset
  episode-sources.ts      — getEpisodeSourcesManifest + singleton + reset
  index.ts                — barrel for the 5-7 getters callers actually import

Rationale: these are meta-data about other data — which mission a photo belongs to, who licensed it, which TTS provider generated which clip. Different shape from "fetch + locale-merge a content JSON." Lumping them in the DAL was a copy-paste accident more than a design choice. Moving them out:

Makes the DAL definition tighter ("fetches + merges content JSON; nothing else").
Each provenance module owns its own singleton + reset hook explicitly. Fixes the latent __resetCache bug (audio + episode-source state was leaking across tests).
Caller impact: 5-10 components (SourceCredit, SourceLogo, LibraryCard, …) import provenance from $lib/data today. They migrate to $lib/provenance in Phase 1's last slice (concurrent with the barrel re-export so no caller breaks).

Q6 · Cache invalidation policy.

Answer: static-forever for v0.7. Document the assumption.

data-core.ts header comment + an inline comment on the cache Map explicitly state: "Static-site assumption — the cache lives for the SPA's lifetime. No TTL, no LRU. At ~100 distinct URLs (mission JSONs + fleet + planets + station + provenance), the memory footprint is bounded by the static asset count. Revisit if we ever go SSR or per-user state."

Phase 1 also exports an unused-by-default cache.size accessor that the rendering Debug tab (#334) can surface as a follow-up — gives visibility without committing to a policy.

Q7 · Cache as footgun — accessor pattern.

Answer: yes, see Q1. The getCached<T>() API hides the Map. Domain modules can't re-declare it because they don't have access to the underlying instance. No import { cache } from './data-core' anywhere in the codebase by design.

New finding — the cross-domain gallery edges

The audit surfaced one architectural decision the issue didn't pre-answer: gallery code is the only place where domain modules call into each other.

Specifically: getMoonSiteGallery and getMarsSiteGallery (both in surface-site) call getMissionGallery + getFleetGallery for fallback. Naive per-domain split puts surface-site upstream of missions + fleet, which forces a one-way DAG that's coherent but constraining.

Answer: extract a separate galleries-data.ts module that owns every gallery getter:

getMissionGallery, getFleetGallery, getPlanetGallery, getSunGallery, getEarthObjectGallery, getMoonSiteGallery, getMarsSiteGallery, getSmallBodyGallery, getSatelliteGallery, getBeltGallery, getSiteStory
getCategoryGallery (the shared category-walker helper)
The loadHeroOverrides + applyHeroOverride integration with $lib/image-hero

Galleries are a vertical concern in the codebase: they all share the same walker pattern (category index → per-id JSON → hero-override application → mission/fleet fallback). They're tied together by the image-hero pipeline more than by their owning subject area.

After extraction:

Domain modules (missions, fleet, surface-site, …) have zero cross-domain calls.
Galleries depends on data-core + $lib/image-hero. Nobody depends on galleries.
The call graph becomes a clean directed tree rooted at data-core.

Module breakdown (final, 9 modules + `$lib/provenance/`)

New module	Owns	Approx LOC
`data-core.ts`	`getCached<T>()`, `clearAllCaches()`, `withLocaleOverlay<T>()`, internal cache	~80
`missions-data.ts`	`getMissionIndex`, `getMission`, `getMissionsForLibrary`, `MissionFilter`, `filterMissions`, `getPorkchopGrid`	~200
`fleet-data.ts`	`getFleetIndex`, `getFleet`, `getFleetByCategory`	~150
`science-data.ts`	`SCIENCE_TABS`, `getScienceTab`, `getScienceSection`, `getScienceLanding`, `getScienceTabIntro`	~120
`surface-site-data.ts`	`planets`, `getPlanets`, `earthObjects`, `getEarthObjects`, `moonSites`, `getMoonSites`, `marsSites`, `getMarsSites`, `getMarsTraverse`, `getSun`, `getRockets`, internal `surfaceHotspotsSidecar` + `hotspotMetadataOverlay`	~400
`station-data.ts`	8 `getIss` getters + 8 `getTiangong` getters	~250
`scenarios-data.ts`	`getScenario`	~30
`galleries-data.ts` (NEW — see "Cross-domain gallery edges" above)	All gallery getters + `getCategoryGallery` + `getSiteStory` + image-hero integration	~350
`small-bodies-data.ts`	`getSatellites`, `getSatelliteI18n`, `getBelts`, `getBeltI18n` + their gallery getters move to `galleries-data.ts`	~150
`$lib/provenance/` (5 files — see Q5)	All provenance + source-logos + audio + episode-source manifests + their 4 singletons + reset hooks	~250
`data.ts` (thin barrel)	Re-exports of the public API across the 9 modules	~80

Net: ~2,060 LOC across 11 files (vs 1,565 LOC in one file today). The 25% overhead is the per-file header comments + module barrels + the new withLocaleOverlay helper. Worth it for the readability win.

Slice plan — Phase 1

10 commits, one per logical extraction. Suggested order (smallest/least-connected first):

Slice	Commit	What lands
1	`feat(data-core)`	`data-core.ts` — `getCached<T>` + `clearAllCaches` + `withLocaleOverlay<T>`. `data.ts` switches to use them (private — no public-API change yet)
2	`refactor(scenarios)`	`scenarios-data.ts`. `data.ts` re-exports.
3	`refactor(science)`	`science-data.ts`.
4	`refactor(station)`	`station-data.ts` (largest single move; isolated).
5	`refactor(missions)`	`missions-data.ts`.
6	`refactor(fleet)`	`fleet-data.ts`.
7	`refactor(surface-site)`	`surface-site-data.ts`. (After slices 5+6 land so surface-site can no-op import them later if needed.)
8	`refactor(galleries)`	`galleries-data.ts` — extracts gallery getters from surface-site + missions + fleet + small-bodies, including the cross-domain fallback logic.
9	`refactor(small-bodies)`	`small-bodies-data.ts`.
10	`refactor(provenance)`	`$lib/provenance/` (5 files). `data.ts` re-exports for back-compat. Last slice — most state-heavy + the latent-bug fix.

After all 10: data.ts is a pure barrel of re-exports. 37 callers untouched.

Each slice runs:

npx svelte-check --tsconfig ./tsconfig.json     # 0 errors
npx vitest run src/lib/data.test.ts             # all pass after each slice
npx vitest run                                  # full suite — 0 regressions

Plus browser-verify on the routes most affected by the touched domain (e.g. /missions after slice 5; /iss + /tiangong after slice 4; etc.).

Risks

Module-state leaks across slices. When a singleton moves out of data.ts into its domain module, its reset hook must be added to clearAllCaches's umbrella. Miss it and __resetCache()-relying tests get stale state silently. Mitigation: every per-domain module exports a _reset() for tests; clearAllCaches calls every one. Codify as the slice template.
Import cycles between domain modules. Eliminated by extracting galleries-data.ts (see the cross-domain finding above). Audit the graph after each slice to confirm no new edge sneaks in — madge --circular src/lib/ in CI catches it.
Vite + dynamic imports. $lib/image-hero is imported dynamically (lines 837 + 1198 of current data.ts). Moving the call sites into galleries-data.ts keeps the dynamic-import pattern; Vite handles it the same way. Verify with a production build (npm run build) after slice 8.
Caller miss in the barrel. A re-export typo in data.ts post-split fails typecheck (most cases) but not always at runtime (interfaces vs values). Mitigation: the existing 988-line data.test.ts exercises 80+ public APIs; running it after every slice catches a dropped re-export instantly.
Latent __resetCache bug fix changes test outcomes. Two known singletons (audioProvenance, episodeSources) currently don't get reset between tests. After slice 10 (provenance) lands, the umbrella reset becomes complete. If any test was silently relying on stale state, it'll surface as a new failure when slice 10 lands. Treat that as a discovered bug, not a regression — fix the test, not the umbrella.

Acceptance

[ ] All 7 Q-answers above ratified by @chipi at draft review.
[ ] Galleries-extraction approach (option A) ratified by @chipi.
[ ] Phase 1 — 10 slices land on main; each independently revertable.
[ ] data.ts is a pure barrel of re-exports after slice 10.
[ ] npx svelte-check 0 errors.
[ ] npx vitest run passes (count grows only by the per-module test exports if we split data.test.ts too — not required).
[ ] npm run preflight exit 0.
[ ] madge --circular src/lib/ reports no cycles.
[ ] Browser-verify after each slice on the routes most affected.
[ ] Bundle measurement on /explore as a follow-up gate for the optional caller-sweep issue.

Open questions (carried into the implementation slices)

Should data.test.ts (988 lines) also split? Not required for Phase 1 — the umbrella test file can stay together and import from the new domain modules. Splitting it later (one test file per domain module) is a clean-up follow-up.
Should __resetCache() keep its underscore-prefixed name post-split? The underscore signalled "internal / test-only." The new umbrella clearAllCaches() could keep the same convention, or graduate to a public name now that the cache contract is locked. Recommend keeping __ for the umbrella; expose a non-underscored clearCache(domain) if a route ever needs targeted invalidation.
Bundle measurement: which tool, which routes? vite-bundle-visualizer is on the dev deps already; baseline on /explore, /missions, /fly, /iss since they're the heavy $lib/data consumers. Run pre-split + post-split.
$lib/provenance/ index module — single barrel or per-concern barrels? Default: single barrel + 5 underlying files. If a caller pattern emerges where one concern dominates (e.g. components only ever need source-logos), split the barrel later.

RFC-028 — data.ts split + DAL architecture review (v0.7) ​

Goal ​

Scope (v0.7) ​

Non-goals (v0.7) ​

Survey — current data.ts shape (verified 2026-06-13) ​

The 7 architectural questions — locked answers ​

Q1 · Per-domain caches vs shared cache singleton? ​

Q2 · Lazy module singletons or query-key pattern? ​

Q3 · Locale-overlay merge — shared transformer or inline? ​

Q4 · Barrel forever, or sweep callers? ​

Q5 · Provenance / source-logos / audio-provenance — DAL or separate concern? ​

Q6 · Cache invalidation policy. ​

Q7 · Cache as footgun — accessor pattern. ​

New finding — the cross-domain gallery edges ​

Module breakdown (final, 9 modules + $lib/provenance/) ​

Slice plan — Phase 1 ​

Risks ​

Acceptance ​

Open questions (carried into the implementation slices) ​