RFC-035 · Prod data freshness — move the launch/TLE refresh from GitHub Actions to an on-VPS cron

Status: Draft · 2026-07-22 · Closes into: RFC-024 (containerized stack + on-demand pipeline runners), ADR-063/064/065 (web container + /data overlay contract) · Related: the GitHub-Pages→docs-only change (separate)

Why this is an RFC. The live product moved from GitHub Pages to the VPS (orrerylearn.com), but the data pipeline didn't move with it. Four things bind here: (1) the 6-hourly refresh currently runs in GitHub Actions and commits to main, which reaches Pages but not the VPS, so prod launch data is silently stale between manual deploys; (2) launch data is served live from the /data overlay (no rebuild) while station TLEs are build-baked into the bundle — two different refresh stories under one cron; (3) the deploy does git reset --hard origin/main and rsyncs build/data/ over the overlay, so any file that is both git-tracked and live-written on the VPS gets regressed on every deploy; (4) retiring the GitHub cron before the VPS replacement is proven would leave prod with no refresh at all. Getting the ownership split, the git-tracking model, and the cutover order wrong means stale or flapping prod data. That's a design decision, not a silent PR.

---

1 · Problem

Two GitHub Actions workflows keep space data fresh:

Workflow	Schedule	Runs	Writes
refresh-launches.yml	17 */6 * * * (every 6 h)	npm run fetch:launches (tsx scripts/fetch-launches.ts)	static/data/launches.json + static/data/launches-historic/<decade>.json
refresh-tles.yml	23 5 * * * (daily)	npm run fetch:tles (node scripts/fetch-station-tles.mjs)	src/lib/satellite/station-tles.json

Both commit the diff to main. That auto-deploys to GitHub Pages, but the VPS deploy (deploy-prod.yml) is workflow_dispatch-only. So orrerylearn.com shows whatever launch data was current at the last hand-triggered deploy — potentially days stale.

Evidence

• deploy-prod.yml has no schedule / workflow_run / push trigger — manual only.
• src/lib/launches/manifest.ts:77 runtime-fetches ${base}/data/launches.json; nginx serves /data/ from the ./static/data bind-mount overlay (ADR-065). So launch data can be refreshed live, without a rebuild.
• scripts/fetch-launches.ts and scripts/fetch-station-tles.mjs use no secrets (Launch Library 2 + Celestrak are public) — verified: no process.env / API-key reads.
• src/lib/satellite/stations.ts:11 does import bundled from './station-tles.json' — the bundle is compiled in at build time. But resolveStationTle (tle-source.ts) live-fetches TLEs from Celestrak at runtime (CORS-open gp.php, cached 1 day in localStorage) and only falls back to the bundle on failure. The AR/sky scenes (ar-scene.ts, sky-scene.ts) use resolveStationTle. So prod TLE freshness does NOT depend on the bundle — the bundle is purely the offline/CORS fallback.
• The pipeline-runner service (compose/docker-compose.prod.yml) mounts ./static/data writable and the repo .:/repo:ro read-only.

2 · Decision

Refresh space data on the VPS itself (option B2): an on-VPS cron invokes the existing pipeline-runner container, writing to the /data overlay that nginx already serves live. Retire the two GitHub refresh crons. This is the design RFC-024 §"pipeline runner" always anticipated ("cron later invokes the container on demand").

Implemented directly in this repo (no separate infra step — the deploy reaches the VPS over Tailscale):

• ops/refresh-prod-data.sh runs the fetch in the container, straight from the upstream sources, and guards against a failed fetch wiping the live file:

  docker compose -f compose/docker-compose.prod.yml --project-directory . \
    --env-file .env --profile manual run --rm pipeline-runner scripts/fetch-launches.ts

  (The image's ENTRYPOINT is npx tsx, so the arg is the .ts entry-point, not npm run ….)
• .github/workflows/deploy-prod.yml runs it once per deploy (prime) and installs an idempotent deploy@ crontab for the 6h cadence.
• The two GitHub refresh workflows (refresh-launches.yml, refresh-tles.yml) are deleted — GitHub is out of the prod data path.

Container fix (load-bearing, verified locally against the published image). The pipeline-runner image bakes node_modules at /repo, but compose bind-mounted .:/repo:ro over it — on the VPS (no host node_modules) that shadows the image's deps and npx tsx breaks; and the :ro mount also blocks the fetch's .launches-cache/ mkdir, which makes it write an empty launches.json. Fix: make the repo bind writable and add an anonymous /repo/node_modules volume so the image's deps survive the bind. Verified: 349 upcoming + 7295 historic entries fetched, no repo pollution.

No secrets to stage (the fetch is keyless). Writes land in /srv/orrery/static/data/ → nginx serves them on the next request. No rebuild, no redeploy, no GitHub.

3 · The two data classes are NOT symmetric

	Launches	Station TLEs
Primary source at runtime	fetch('/data/launches.json') (overlay)	live fetch from Celestrak (resolveStationTle, cached 1 day)
Fallback	—	build-time import './station-tles.json' bundle
Prod staleness if not refreshed	Real — no live source; the overlay IS the data	Minimal — live fetch keeps it fresh; a stale bundle only degrades the offline/CORS fallback

So the genuinely-stale-on-prod problem is launches — B2 solves that cleanly (overlay refresh, no rebuild). TLEs are not actually stale on prod because the runtime live-fetches Celestrak; the build-baked bundle is just the fallback. That reframes the TLE options (this corrects an earlier draft that claimed TLEs go stale on prod — they do not):

• T-none — Leave the TLE bundle build-baked (recommended). It refreshes on deploy, and since it's only the offline/CORS fallback behind a live fetch, deploy-cadence freshness is fine. Just retire the daily TLE GitHub cron (refresh-tles.yml); the fallback stays deploy-fresh. Zero refactor.
• T1 — Migrate the bundle to the /data overlay. Move station-tles.json → static/data/station-tles.json, convert stations.ts's static import to an async fetch, so the VPS cron keeps the fallback daily-fresh and it leaves git. Cleaner/decoupled, but lower value (fallback-only) at real cost: stationTle goes sync→async, rippling into tle-source.ts's fallback path and satellite.test.ts.

Open decision D1 (re-opened after the live-fetch finding): T-none (leave the bundle, retire the TLE cron) vs T1 (migrate the fallback to the overlay). Recommendation flips to T-none — the live fetch already keeps prod TLEs fresh, so the async refactor buys only a fresher rarely-hit fallback.

4 · The deploy-regress trap (load-bearing)

deploy-prod.yml runs git reset --hard origin/main and rsync build/data/ → static/data/. Any file that is git-tracked and live-written on the VPS is reset to the committed version on every deploy. launches.json + launches-historic/* are git-tracked. So once the GitHub cron stops committing fresh data, every deploy would wipe the VPS's B2-fresh launches back to the frozen committed copy.

Fix: treat the live-refreshed files as generated runtime data, not source — .gitignore them (the precedent already exists: static/data/i18n/*.json is gitignored, generated by build-i18n-bundles.mjs). git reset --hard leaves ignored/untracked files untouched, and the deploy seed must exclude them. Concretely add to .gitignore:

static/data/launches.json
static/data/launches-historic/*.json
static/data/station-tles.json        # only under T1

and git rm --cached them so they leave the tree, and add matching --exclude filters to the deploy-prod.yml seed rsync so a deploy never overwrites live data.

Open decision D2 — dev + bootstrap seed: with the files untracked, a fresh clone (and the VPS's first boot) has no launches.json, so /missions/launches renders empty until a fetch runs. Two options: (a) commit a small *.seed.json fixture that dev/bootstrap copies on first run; (b) document "run npm run fetch:launches once after clone" (keyless, ~seconds) + have the VPS cron's first invocation seed it. Recommendation: (a) for zero-friction dev DX; the seed can be a trimmed snapshot refreshed occasionally.

5 · GitHub Pages (related, not this RFC's core)

With the app served only from the VPS, the GitHub-Pages app mirror is a stale, divergent, SEO-duplicate copy. The agreed direction is to make Pages docs-only (keep deploy-docs.yml's VitePress build; retire the app publish in staging.yml). That removes Pages' dependence on the launch data entirely, so it doesn't interact with B2. Tracked as its own change; noted here for coherence.

6 · Ownership + sequencing (rollback-safe)

Split (mirrors the Umami/GlitchTip pattern):

• Infra / VPS provisioning: the cron unit (systemd timer or crontab) that invokes docker compose run --rm pipeline-runner ...; ensuring the image is pulled. Keyless — no secret staging.
• Orrery repo (this work): .gitignore + git rm --cached the runtime data; exclude it from the deploy seed/reset; T1 refactor (if chosen); Pages docs-only; retire the GitHub crons; this RFC + an ADR to lock the outcome.

Cutover order — never leave prod without a refresh:

1. Land the repo changes (gitignore + deploy-protect + dev seed + T1 if chosen). These are inert until the crons change.
2. Infra provisions the VPS cron; verify it writes a fresh static/data/launches.json on the VPS and nginx serves it.
3. Only then retire the GitHub crons — disable the schedule: triggers first (keep workflow_dispatch as a manual bridge), delete later once B2 has a few green cycles.

Rollback (<5 min): re-enable the GitHub schedule: triggers (kept as workflow_dispatch), git add the data files back. The committed-data model is fully restored.

7 · Alternatives considered

• A — Auto-redeploy the whole build every 6 h (add workflow_run/schedule to deploy-prod.yml). Rejected: a full rebuild + rsync for a data-only change that the overlay serves live without one. Wasteful CI + churn.
• B1 — Keep the GitHub cron, add a step that rsyncs the refreshed /data to the VPS overlay over Tailscale. Works and is light, but keeps the refresh dependent on GitHub Actions. Rejected per the goal of a self-contained prod that refreshes itself (same rationale as self-hosting logs/errors/analytics).
• B2 — On-VPS cron drives pipeline-runner → overlay (chosen). Fully self-hosted, no GitHub dependency, no rebuild, reuses existing compose plumbing.

8 · Non-goals

• Any change to the fetch scripts' data sources or schemas.
• Moving the other pipelines (images, provenance) to the VPS — out of scope; they stay CI/manual.
• The GitHub-Pages→docs-only change itself (referenced in §5, tracked separately).

9 · Decisions (resolved)

• D1 — TLEs: T-none. The browser live-fetches TLEs from Celestrak (primary); the build-baked bundle is only the offline fallback. So no VPS TLE refresh is needed — refresh-tles.yml is deleted and the fallback stays deploy-fresh. No async refactor.
• Launches gitignore/deploy-regress: not needed with this implementation. The VPS fetch overwrites the live launches.json in place, and the deploy's git reset --hard re-confirms the committed copy (still fine as a seed) before the prime-run refreshes it. Revisit only if we later stop committing launches.json entirely.
• D2: dev/bootstrap seed for the now-untracked data — (a) committed *.seed.json fixture vs (b) dev-fetch-once. Rec: (a).