ADR-135: v2.4 — GI route-and-tag insights + KG Voice node (one coordinated schema bump)

• Status: Partially superseded — #1191 (route-and-tag) Accepted & implemented; #1220 (Voice node) REVERSED 2026-07-28, see the note below.
• Date: 2026-07-28
• Authors: Marko Dragoljevic
• Related issues: #1191 (GI route-and-tag), #1220 (KG Voice node — reversed), #1193 (read-side leak, done)
• Design source: ADR-143 — the reprocess methodology; v2.4 is one applied single-variable step

Reversal note (2026-07-28) — #1220 Voice node dropped. After measuring the LLM-labeled corpus, the operator decided unresolved diarization voices are noise to cut, like commercials — not an entity to model. Measured: on the 9-episode LLM-labeled slice, 44 real graph voices → 20 named; the ~24 unnamed were concentrated in narrative/documentary shows (15 in one Planet Money episode), and zero Voice nodes actually reached the graph because _resolve_quote_speaker (the #1167 guard) already drops voices carrying a voice_type. So Voice was a never-populated node type. The goal is instead to drive unrecognized voices toward zero via better labeling, and drop the residue. What was reverted: the KG/GI Voice node type (schema back to KG 2.0; GI voice_node → reserved speaker_node), the person_node_type_for_name write-side, migrate_kg_document_v2_1, and the Voice-retype in migrate_gi_document_v3_1. Decoupling that matters: the player transcript keeps every segment attributed to its "unknown/unidentified speaker" friendly label — only the graph stops minting a node. #1191 (route-and-tag) is untouched and stays.

Context & Problem Statement

v2.4 in the prod-reprocess arc is "GI/KG optimizations". It is TWO changes that both alter the stored artifact shape, so per the arc's core rule they must land in one rebuild (or the corpus is reprocessed twice):

• #1191 — the GI pipeline hard-caps insights (GI_MAX_INSIGHTS_CEILING, now 200 after an interim duration-scaled stopgap), truncating real gated-good insights into the corpus. The design (GI_WHAT_TO_SURFACE.md) mandates: the pipeline ranks and tags, it never truncates — "first N" is a view-time decision.
• #1220 — unresolved diarization voices (SPEAKER_NN) are written as first-class Person nodes and only filtered at read time (per-consumer guards, leaky). They must be a distinct node type so Person queries never see them by construction.

Recon (2026-07-28) established the coupling: the "unnamed-voice → Person" bug exists in both the KG writer (kg/pipeline.py) and the GI writer (gi/pipeline.py::_attach_person_for_quote), fed by the same graph_id_utils helpers; the value gate already computes per-insight tiers (FILLER/MINOR/USEFUL/CORE) and then discards them; and gi.schema.json already reserves an unused Speaker node type that collides with #1220's Voice.

Decision

1. One coordinated schema bump

• GI: gi.schema.json schema_version 3.0 → 3.1 (additive Insight fields).
• KG: kg.schema.json schema_version 2.0 → 2.1 (new Voice node type).
• Both keep the existing strict-gate + migration-script discipline (gi/schema.py, kg/schema.py).

2. #1191 scope — store rank/tier/tag, no truncation (rubric research DEFERRED)

• Add to the Insight node (additive, additionalProperties:false extended): rank (float), tier (int, the value-gate tier we already compute — stop discarding it), routing_tag (surface | connect | drop), salience (float, the composite score).
• Preserve the value-gate tier through value_gate.py → contracts.py → schema instead of collapsing it to a keep/drop boolean.
• Remove the pipeline truncation slices (gi/pipeline.py _resolve_insight_specs / build_artifact prefilled cap) so every gated-good insight is stored. gi_max_insights + GI_MAX_INSIGHTS_CEILING become extraction/token-budget safety only, never a corpus cutoff; the per-provider min(…, ceiling) stays as that safety bound.
• Rank = composite of signals that ALREADY exist: tier × speaker-role (gi/speakers.py) × grounding-strength (roll up per-quote qa_score/nli_score from gi/grounding.py) × corpus novelty (extend the gi/about_edges.py embedding scaffold). "First N" moves to the viewer.
• NON-GOAL (deferred follow-up, tracked on #1191): the SURFACE/CONNECT/DROP taxonomy overhaul, the insight_type enum change (STANCE/ARGUMENT/EXPERIENCE/EVENT), and the judge-rubric/model research. The design doc itself lists these as unresolved (44.6% misclassification). They do not change the artifact shape; routing_tag is stored now, its classifier is refined later.

3. #1220 scope — Voice node in BOTH layers

• Add a Voice node type to kg.schema.json (new $defs + oneOf entry) and repurpose the reserved Speaker type in gi.schema.json as Voice (resolve the collision — one name for the concept across sibling schemas).
• Branch both write sites off the shared placeholder check (graph_id_utils.is_bare_speaker_label / the person:speaker-NN id shape): emit type = "Voice" in kg/pipeline.py::_typed_person_org_node and in gi/pipeline.py::_attach_person_for_quote instead of Person. SPOKEN_BY / turn edges keep pointing at the Voice node (KG edges are endpoint-type-agnostic; confirm GI edge validators accept it).
• Exclude Voice from graph_id_utils.PERSON_ORG_NODE_TYPES ({Entity,Person,Organization}) — a Voice is not a resolvable identity, so it must not participate in person/org edge-building, entity rollup, or cross-episode embedding (kg/corpus.py, gi/relational_edges.py).
• Repurpose is_unresolved_speaker_placeholder (the #1193 read guards) as legacy defense for un-migrated data, not the primary mechanism.

4. Migration + fixtures + rebuild

• Two idempotent migrations in the corpus-upgrade framework (the m0006 template): retype person:speaker-NN Person → Voice in KG artifacts and in GI artifacts (skip already-Voice nodes). Validate on a prod-v2/prod-v3 copy before shipping.
• Regenerate the affected golden fixtures (39 files under tests/fixtures/viewer-validation-corpus/v3, ~820 person:speaker- occurrences).
• The v2.4 corpus rebuild reprocesses with the new schema; existing corpora are brought forward by the migrations. Full ci-fast incl. the KG/viewer stack-test.

5. Eval + viewer (follow-through, same arc)

• Per-tier / per-routing_tag metrics; re-baseline avg_insight_nodes (counts change with no cap).
• Viewer gains the ranked "first-look + expand" view (builds on #1048 ranked-topic overview).

6. Consumer wiring — per surface (GATED on the eval verdict)

#1191 is write-side-only until this is done. The rank/tier/routing_tag/salience fields are stored, but as of the 2026-07-28 revert audit no consumer reads them — graph queries, search ranking, the operator viewer, and the player all ignore them. So today the corpus pays the cost of de-truncation (more nodes/vectors, longer flat lists) without the benefit of sortability. Quality floor is unchanged (gi_value_gate_min_tier=2), so the extra insights are USEFUL/CORE, not filler.

Shipped default (one rule, everywhere): sort by salience desc → cap at a default N → "show more" reveals the rest → exclude routing_tag=drop. All read-time, no schema change. This is safe to ship before the eval verdict because salience-sort is never worse than today's extraction order and the cap bounds it; the eval only tunes N and decides whether to trust salience further (e.g. hiding connect on the operator side). The default N is gi_surface_default_limit (config, default 8), overridable per surface. Cap unit: per topic for player/viewer lists, per episode on the episode detail view.

Surface	Sort	Filter	Default cap	Notes
Player (consumer)	rank (salience)	surface-tagged only	6	never a wall; connect is corpus plumbing, hidden from listeners
Operator viewer	salience desc	all tiers + routing_tag filter chip	12 + expand	power user; completeness on demand (builds on #1048)
Search	salience rerank boost	drop excluded	no hard cap	query-driven; boost, don't truncate
Graph API	salience desc	default view = surface	8	briefing by default, connect on expand

Eval still gates the trust decisions, not the default itself. The semantic A/B (scripts/eval/insight_salience_eval.py: salience-topN vs extraction-topN, cross-vendor Claude judge) confirms whether salience tracks independent quality. If it does not, fix the ranking (the salience = f(tier, grounded, surfaceable) formula or the value-gate tier) before leaning harder on it (e.g. before hiding connect).

Eval VERDICT (2026-07-29, 100-ep / 866 insights, claude-sonnet-5 judge). Salience does not track quality (Spearman 0.16); the value-gate tier that feeds it barely discriminates (CORE 3.06 vs USEFUL 2.95 on 1–5). So: the salience-sort default stays (harmless — +0.12 at top-3, ~flat after), but the search boost and operator routing_tag filter chip stay deferred indefinitely — do not gate/boost on a 0.16 signal. The one measured win is de-truncation bounded to ~8 (ranks 6–8 as good as top-5; tail beyond 8 weaker → gi_surface_default_limit=8, player fold 6→8). Consequence: the "rubric research" below is upgraded from DEFERRED to REQUIRED — a discriminating tier is the prerequisite for ranking to ever be worth wiring, and this eval is the evidence. Insight quality (extraction + gate), not order, is the next real lever (overall 3.0/5).

Consequences

• Artifact shape changes → corpus migration + fixture regen + eval re-baseline; this is why v2.4 is one rebuild.
• Voice typing is durable — Person queries never see unresolved voices by construction; the ~13 read-side guards become belt-and-suspenders.
• Deferring the rubric keeps v2.4 shippable: the corpus gets the ranked/tagged/untruncated shape now; the classifier that fills routing_tag well is a separate research cut, no second rebuild (the field already exists).

Non-Goals

• The GI classifier-rubric / judge-model research and the insight_type taxonomy change (#1191 follow-up). The DGX-LLM swap (v2.5). LoRA / fine-tuning (out of scope, autoresearch closed).