Skip to content

ADR-103: Deterministic connectivity under LLM-free profiles

  • Status: Accepted
  • Date: 2026-06-24
  • Authors: Marko Dragoljevic, Claude (Opus 4.7)
  • Related ADRs: ADR-101 (strict v2.0 / v3.0 contract these post-passes target); ADR-102 (audit trail for sweep-style mutations).
  • Closes: #1058 — connectivity over a real pipeline-synthesized corpus in CI without a cloud LLM.

Context

The exploration / relational connectivity surfaces (person→topics, co_speakers, topic→related_topics, cross_show_synthesis, entity_neighborhood, etc.) need a corpus that actually carries Insights + Topics + the right edges. We had five layers of coverage but one hole:

  • ✅ Logic — unit tests on hand-built graphs.
  • ✅ Pre-seeded data — checked-in production-shaped fixture + grounded on-disk repro corpus.
  • ✅ Viewer rendering — mocked Playwright e2e.
  • ✅ Stack-test route wiring — /api/relational/* reachable + well-formed empty payloads over the airgapped pipeline output.
  • ❌ Real pipeline → connected corpus → live server, in CI.

The last bullet is the gap. In CI the only profile is airgapped_thin (cloud_thin would call a paid LLM — [[feedback_no_llm_in_ci]]). Airgapped's no-LLM summary provider can't extract typed entities, so its synthesized corpus has empty connectivity. Stub mode was explored and dropped.

The operator framing: close the gap without introducing stub mode, with the existing dependency set (no new heavy models).

Decision

Close the gap with deterministic local enrichment on top of the existing airgapped path. Six chunks land on feat/1058-airgapped-deterministic-connectivity:

Chunks 1–3 — synthesize the missing structure

  1. KG ORG post-pass (kg/ner_postpass.py, kg_organizations_use_ner: true for airgapped*). Runs spaCy NER over GI Insight texts and adds Organization nodes for every distinct ORG span. Idempotent. Conforms to ADR-101's strict Organization shape (id: org:{slug}, properties.name, properties.role: "mentioned"). False-positive guard: drops spans <3 chars + spans that are all-numeric.

  2. GI MENTIONS_ORG via the existing NER pass (gi/relational_edges.py). The typed-MENTIONS post-pass (#1076 chunk 4-A) already handled the MENTIONS_ORG vs MENTIONS_PERSON selection via the entity-index kind field; the only change is the spaCy label filter (PERSON | ORG). Reuses the shared-surname disambiguation guard.

  3. Cross-show Topic clustering (kg/topic_clustering.py, kg_topic_corpus_clustering: true — captured into the eval fingerprint). Walks every KG, gathers (episode_id, podcast_id, label, topic_id, kg_path), embeds via sentence-transformers (all-MiniLM-L6-v2 — same model the ABOUT layer uses), greedy single-link clusters on cosine ≥ 0.75. For each cluster spanning ≥2 distinct episodes, adds a synthetic concept:topic-{slug} Topic (is_concept: true) plus RELATED_TO edges from every member. Idempotent.

Surfaced both as a workflow auto-trigger and an operator CLI:

  • Workflow — workflow/orchestration.py Step 16 (after _finalize_pipeline) fires cluster_and_apply_corpus_topics on every corpus run where cfg.kg_topic_corpus_clustering is True. Non-fatal — a clustering failure doesn't bring down a successful run.

  • CLI — operators can re-run / experiment on an existing corpus without a full pipeline pass:

    .venv/bin/python -m podcast_scraper cluster-corpus-topics \
    --output-dir <corpus> [--threshold 0.75] [--min-episodes 2] [--dry-run]

Chunks 4–5 — prove the server / viewer layer renders it

  1. Multi-show pre-seeded fixture (tests/fixtures/connectivity-multi-show/). 3 shows × 2 episodes = 6 episodes, generated by build_fixture.py. Exercises every connectivity surface (cross-show Person, intra-episode co-speakers, per-show Topic + ABOUT, MENTIONS_PERSON + MENTIONS_ORG, cross-show concept-Topic + RELATED_TO, entity neighborhood). The generator IS the contract — byte-equal re-runs, indented JSON sorted keys.

  2. Tier-3 connectivity test (tests/integration/connectivity/test_relational_queries_against_fixture.py). Loads the fixture via the production CorpusGraph and asserts every relational query surface returns non-empty data for at least one valid input. The same code paths the FastAPI server serves.

Chunk 6 — docs / ADR

This file plus a corpus/ontology.md section documenting the post-passes.

Trade-offs accepted

  • spaCy en_core_web_sm precision floor. ORG extraction will occasionally tag organisations that aren't (or miss real ones). The FP guard + idempotency means false orgs accumulate in the KG, but downstream relational queries cap at k=20 and rank by occurrence — noise is bounded.
  • MiniLM at threshold 0.75. Higher → fewer cross-show concept-Topics; lower → spurious cluster joins. 0.75 matches the existing topic_clusters_default_0_75 preset that's been live in production since v2.6.0; same default carries forward here.
  • Idempotent but not migratory. Re-running the post-passes over an already-enriched corpus adds nothing. Mutating sweeps use the ADR-102 _retro_audit pattern; the workflow path is greenfield writes (no audit needed).

Alternatives considered

  • Enrich stub mode. Rejected by operator — would require inventing fake Insights / Topics that don't reflect real content.
  • Local Topic derivation only (skip ORG). Would close the cross-show synthesis dimension but leave Organization / MENTIONS_ORG empty, so insights_about(org) and the Organization side of entity_neighborhood stay broken under airgapped.
  • Multi-show pre-seeded fixture only. Backstops the server + viewer layer (chunks 4-5 cover this) but doesn't close the pipeline gap — airgapped corpora would still be empty under a real CI pipeline run.

The chosen "pipeline + fixture" hybrid is direction (A + C) from the issue body — both layers caught, each by a different test.

Consequences

Positive:

  • Airgapped CI now carries real connectivity data for every surface the relational query layer + viewer consume.
  • cross_show_synthesis and related_topics across feeds work on local-only profiles for the first time.
  • No new dependencies — uses spaCy + sentence-transformers that airgapped already loads for speaker detection / ABOUT edges.

Negative:

  • Three new opt-in cfg flags increase the config surface; the registry-drift overlay test (_OVERLAY_EXPECTED) pins them per profile.
  • Workflow gains a corpus-level post-pass step (Topic clustering) that runs after every episode is processed — modest wallclock cost, dominated by the existing sentence-transformers model load.

Neutral:

  • The fixture generator (build_fixture.py) is the contract for what connectivity surfaces matter; edits there need a re-run before commit, and a fixture-shape contract test (test_multi_show_fixture.py) catches drift if the JSON gets out of sync.

Validation

  • 26 unit tests on kg/ner_postpass, 18 unit tests on kg/topic_clustering, 4 new ORG-flavour tests on the existing relational_edges NER pass.
  • 6 profile-overlay tests + drift-test pins for the three new flags.
  • 9 fingerprint capture tests across the three new flags (3 per flag: captured-when-set / omitted-when-absent / flip-produces- different-hash).
  • 10 fixture-shape contract tests + 9 Tier-3 connectivity surface tests on the multi-show fixture.
  • 8 smoke tests for the cluster-corpus-topics CLI.
  • make ci-fast green (final state).

Implementation references

  • src/podcast_scraper/kg/ner_postpass.py — ORG node post-pass.
  • src/podcast_scraper/kg/topic_clustering.py — corpus-level clustering.
  • src/podcast_scraper/search/cli_handlers.py::parse_cluster_corpus_topics_argv and run_cluster_corpus_topics_cli — operator-facing CLI.
  • src/podcast_scraper/cli.py — cluster-corpus-topics command dispatch.
  • src/podcast_scraper/gi/relational_edges.py:_apply_ner_mentions_pass — extended PERSON | ORG filter.
  • src/podcast_scraper/workflow/metadata_generation.py — KG ORG post-pass wiring next to the existing GI MENTIONS post-pass.
  • tests/fixtures/connectivity-multi-show/build_fixture.py — fixture generator + connectivity contract.
  • tests/integration/connectivity/ — fixture-shape contract + Tier-3 connectivity surface tests.