RFC-029 — Image staging ground · fetch→stage→approve→promote (v0.7)

Status · Shipped (2026-06-27) Date · 2026-06-24 Target · v0.7 Tracked by · GH issue #363

Why this is an RFC. /credits is supposed to be the honest bill-of-materials for the imagery Orrery ships. Today it lists every downloaded image — image-provenance.json has 3,512 entries with zero display filtering, so ~828 vision-rejected images (and an unknown number of orphans) are credited and shipped despite never rendering. Fixing the credits page alone leaves the rejected files in the deployed bundle (bloat + distributing CC-BY assets we don't credit-in-context). The durable fix changes the pipeline's shape: downloads land in a staging ground, only approved images reach the shipped tree, and provenance walks the shipped tree only. That's an architecture change across the fetchers, the provenance build, the deploy/docker steps, and a one-time migration — locked here before the slices start.

Gating sentence: Locks the image staging-ground pipeline — static/images/_staging/ (gitignored) receives every new fetch, vision scores it, a human approval in /dev promotes it to the shipped static/images/ tree, build-image-provenance.ts walks the shipped tree only so /credits == shipped == displayed, build/deploy/docker prune _staging/, rejected assets stay in staging behind an image-tool hard-prune, and a one-time migration relocates today's vision-rejected-and-undisplayed images out of the shipped tree — tracked by issue #363.

Goal

credited == shipped == displayed. A new image cannot reach the deployed bundle (or /credits) without passing curation and an explicit human promote; rejected images are quarantined in a gitignored staging ground, never shipped, never credited, recoverable, and hard-prunable on demand.

Scope (v0.7)

• static/images/_staging/ staging ground (gitignored) + the fetch / score / promote flow.
• build-image-provenance.ts walks the shipped tree only (skips _staging/).
• Build/deploy/docker exclude _staging/.
• One-time migration of the current corpus's undisplayed rejects into staging.
• Image-tool prune-staging (hard delete) option.

Non-goals (v0.7)

• Re-curating the existing displayed corpus. Anything linked/displayed today is auto-approved and stays in the shipped tree untouched (decision 2).
• Changing the vision-scoring model or thresholds (score-images.ts unchanged).
• A new public review UI — promotion drives off the existing /dev curation surface.

Survey — verified current state (2026-06-24)

• image-provenance.json: 3,512 entries, built by build-image-provenance.ts walking every on-disk file under static/images/** (curated walkers + a direct-other fallback). No display/approval field.
• /credits (src/routes/credits/+page.svelte): renders provenance.entries verbatim — no vision/rejection/displayed filter.
• image-vision.json: 2,066 scored → 1,238 approved (rejected_by == null), 828 rejected (374 score-below-threshold, 276 category-diagram, 178 category-people). The UI pickers honor rejection; /credits does not.
• image-curation.json: empty / unwired.
• ~1,446 provenance entries are unscored (logos, SVG diagrams, the flat-family covers/heroes, textures, panoramas) — legitimately used; they must stay.
• adapter-static copies all of static/ into build/; docker's nginx serves the host build/. So excluding staging from deploy needs a post-build prune (or staging never committed + pruned anyway).
• Existing pruning machinery to lean on: scripts/prune-orphan-images.ts, scripts/prune-image-slots.ts.

Decisions (locked with Marko)

1. Staging is gitignored — unapproved downloads never bloat the repo; only the shipped tree is version-controlled. Trade-off: rejects aren't preserved across machines, but they're re-fetchable.
2. Forward policy: vision-score → staging; human approve in /dev → promote to the shipped tree. The policy is not retro-applied — every image linked/displayed in the current corpus is auto-approved and left in place.
3. Rejected images stay in staging for re-review; an image-tool prune-staging option hard-deletes them when the curator decides.
4. Durable approved allowlist — static/data/image-approved.json is the sticky source of truth for "used/approved," independent of vision scores. /credits + provenance derive from it; re-running provenance (new files, walk changes, re-scores) can never drop an approved image. A human approve appends; entries are never auto-removed. This is the answer to "don't accidentally drop a used image."
5. Manual approve / skip / prune — the image-tool reviews the staging folder and lets a human approve any image (→ moves to main + appended to the allowlist), skip it (leave staged), or prune staging (hard delete) — manual approval overrides vision regardless of score.

image-approved.json — the durable source of truth

Shape: { version, generated_by, note, count, approved: string[] } — every approved/used image path. Derived (not hand-frozen) by scripts/audit-image-usage.ts --write-approved: the allowlist is exactly the set of shipped images the usage audit certifies as displayed (referenced in static/data/src, within a gallery count-cap, a hero slot, a body surface, a hotspot tier, or a card). Re-derive after fetches land; because the audit only ever certifies used images and human-approves append, a re-derive can never drop a still-displayed image. Provenance and /credits are an intersection of (on-disk shipped tree) ∩ (approved list) so neither a stale walk nor a re-score can drop a used image.

Empirical finding from the migration (2026-06-24): the displayed corpus was already ~99.9% honest. "Vision-rejected" never controlled gallery display — gallery count-caps do — so the ~828 vision-rejected images that show in /credits are, with rare exception, legitimately displayed. The audit found only 7 true orphans (surface-gallery slots above their count-cap: mars-sites/mangalyaan 06–07, mars-sites/mars-express 06, moon-sites/apollo15 06–09); these were staged out. The lasting value of this RFC is therefore the forward staging flow + the durable allowlist + the committed audit gate, not a mass relocation.

Slice plan — ✅ SHIPPED (2026-06-27)

Slice	Work	Status
1	static/images/_staging/ + .gitignore; build-image-provenance.ts skips _staging/	✅ b1488b8d7
—	Decision 4 — build-provenance intersects image-approved.json (scoped to /images/; logos/textures bypass); /credits derives from the allowlist	✅ fa5f1847e
2	Migration — moot: audit proved the corpus was ~99.9% honest; only the structural mars-site orphans were staged out (no mass relocation)	✅ (folded into reconciliation)
3	Fetchers stage by default — scripts/lib/staging.ts toStagingDest() redirects gallery-slot writes in fetch-assets.ts; gallery-count writes skipped when staging; --no-stage opt-out	✅ 202327ff0
4 + 5 (Decision 5)	/dev/staging review surface (rich grid, filters, bulk-mark) + api/+server.ts "Apply marked actions" → promote (move + append allowlist + bump contiguous gallery count) / prune	✅ 6b3eee98e
5	Deploy prune: post-vite build scripts/prune-build-staging.mjs rm's build/images/_staging + asserts none survives	✅ 0a91d6d63
6	Hard-delete: the /dev/staging Prune action (no separate CLI needed)	✅ 6b3eee98e

Notes vs the original plan: promote + prune are one "Apply marked actions" POST in the review surface (no separate promote-images.ts / prune-staging CLIs). Decision 4's intersection is scoped to /images/ so /logos/ + /textures/ (never staged) stay credited. Promote bumps each id's gallery count to its contiguous main-slot count, keeping galleries gap-free. Crediting a promoted image still needs npm run build-image-provenance (use --offline to skip Wikimedia).

Migration — the safe move-list (Slice 2)

Move = image-vision rejected_by != null AND not displayed, where displayed means referenced by an explicit display path. Guards:

• Exclude hero slots (…/01.jpg + variants) — heroes, never moved.
• Exclude images explicitly path-referenced in static/data/** excluding the manifests themselves (image-vision.json, image-provenance.json, image-curation.json, *-galleries.json count maps). The first naive pass scanned the manifests and matched everything → 0 to move; the corrected pass scans only display-reference files (hero-overrides resolved to paths, science photo.src, surface-site photo refs).
• Output a dry-run report (count + MB) under docs/provenance/ and require eyeball sign-off before the move.

Risks

• False-move (relocating a displayed image): mitigated by the explicit-reference guard + slot-01 guard + dry-run sign-off + in-browser gallery check after the move.
• Deploy leak (staging shipped): mitigated by gitignore and the post-build prune (belt + suspenders).
• Cross-machine fetch divergence (staging gitignored): acceptable — staging is local curation scratch; CI/cron fetchers stage then promote within the same run.

Acceptance

• /credits entry count == images actually rendered (no vision-rejected, no orphans).
• grep -r _staging build/ is empty after a production build.
• A new fetch lands in _staging/ and cannot reach build/ without promote-images.ts.
• Each slice independently revertable + browser/validate-verified.

Open questions (carried into the slices)

• Does promote-images.ts drive purely off an /dev-written approval list, or also auto-promote vision-approved on first score? (Lean: explicit approval only, per decision 2.)
• Should prune-staging also drop the corresponding image-vision.json entries, or keep the scores for re-fetch dedupe? (Lean: keep scores; content-hash cache already dedupes.)