# F9 — Entity-Anchor Query Expansion: Closure (2026-05-27)

> Status: closed. q15 zero-recall diagnostic chain (F1 → F2 → F4 →
> F5 → F6 → F7 → BL-9 → F9) ends with `path_recall = 1.0` on q15
> after enabling `JAMES_ENABLE_ENTITY_ANCHOR=1`. Companion to the
> LEO L.D result doc (`v3prime-leo-evidence-scope-result.md`,
> measurement substrate) and v0.4.0 release notes.

## Headline

`q15 ("David Soria Parra가 누구야?")` is the canonical proper-noun
cross-lingual retrieval failure that surfaced during step7 v4 path-GT
measurement (Idea 1, PR #530). The full diagnostic chain attributed
the failure across eight cycles, ruling out the intent classifier,
the policy gate, retrieval stochasticity, the rerank stage, and even
the embedding model — and finally pinned the root at **query
expansion**: any 1024-dim multilingual pooling embedding (bge-m3
included) cannot match a bare proper-noun query against a
concept-dense chunk vector unless the corpus anchor ("MCP", "Model
Context Protocol", "Anthropic") is materialized into the query.

F9 is the fix:
- **F9.1** (PR #545) — `scripts/research/query_rewriter_concept_anchor_audit.py` proves the LLM rewriter alone adds 0/3 concept anchors on `bare_proper_noun` queries.
- **F9.2** (PR #546) — `core/retrieval/entity_anchor_expander.py` does the deterministic graph-walk: NER → graph relations → top-K co-occurring concept terms → injected as the "anchor" prefix. LLM-free, ~0.1ms, 31 unit tests + 6 audit A/B arm tests.
- **F9.3** (PR #547) — `pipeline.py` STEP 0.5a calls the expander when `JAMES_ENABLE_ENTITY_ANCHOR=1`. 16 wiring tests. Flag default OFF (byte-identical to F9.2 main).
- **F9.4** (this doc) — operator-measured acceptance.

## Diagnostic chain (8 cycles → 1 fix)

| Cycle | PR | What was ruled out | What was promoted |
|---|---|---|---|
| F1 | #527 | bench-suite role gate (external rolled to `handle_chat`) | retrieval-mode harness + JWT bearer |
| F2 | #531 | IntentClassifier (`14/14 = 100% accurate`) | classifier audit canary |
| F4 | #528 | narrow-band scope reachability (structural ceiling at 0.40) | scope formula fix candidate (→ F5) |
| F5 | #529 | k=0 floor (entropy/spread were chroma filler) | aggregate = graph_reach only when ek=0 |
| F6 | #532 | extraction stochasticity (5/5 byte-identical) | chroma rerank / embedding match |
| F7 | #533 | "MCP Model Context Protocol" (EN concept) wins top-1 / "David Soria Parra" any form does not | proper-noun-mediated retrieval = systemic MiniLM weakness → BL-9 swap |
| BL-9 | #534 + #536 | bge-m3 swap active (concept_side score 0.85 → 0.76 differential confirmed) | q15 `name_only` still fails → root is query expansion, not embedding |
| **F9** | #545/#546/#547 | — | EntityAnchorExpander + STEP 0.5a pipeline wiring |

The cost-of-diagnosis pattern matters here: each audit took ~30
minutes; "throw a fix" reflex would have wasted days. The lesson
already lives in `feedback_intent_classifier_audit_clean.md`. F9
is its second confirmation.

## F9.4 acceptance measurement (2026-05-27 23:12)

```
git_sha:  f2c088f
bench:    JAMES_BENCH_BEARER=<employee JWT> python scripts/bench.py --suite=step7 --mode=retrieval
env:      JAMES_ENABLE_ENTITY_ANCHOR=1
          JAMES_EMBEDDING_MODEL=BAAI/bge-m3
          JAMES_ENABLE_QUERY_REWRITE=1
result:   reports/research-runs/step7-bench-20260527_231211.json
```

### Path-recall aggregate

```json
{
  "queries_with_expected_path": 4,
  "mean_path_recall": 1.0,
  "queries_at_full_recall": 4
}
```

| q | category | expected_path nodes | hits | path_recall | unified_score |
|---|---|---|---|---|---|
| q2  | retrieve | Anthropic | 1/1 | **1.00** | 0.862 |
| q3  | relation | Anthropic, Claude | 2/2 | **1.00** | 0.902 |
| q4  | relation | BlackRock, 비트코인 ETF | 2/2 | **1.00** | 0.930 |
| **q15** | **person** | **David Soria Parra, MCP** | **2/2** | **1.00 ⭐** | — |

q15 baseline (pre-F9 / pre-BL-9): `path_recall = 0.0` on the same
fixture (Idea 1 first measurement, PR #530).
q15 with `JAMES_EMBEDDING_MODEL=BAAI/bge-m3` alone (post-BL-9, pre-F9):
still `path_recall = 0.0` on `name_only` shape (BL-9 acceptance run,
PR #536).
q15 with all three flags active (this run): `path_recall = 1.0`,
answer correctly identifies "MCP 설계자".

q1 timed out (`RAG가 무엇인가?`, 120s budget exhausted) —
cold-cache phenomenon unrelated to F9. The other 11 queries
returned `ok` without expected_path GT annotations, so they don't
contribute to the aggregate; their answer_len / graph_paths /
unified_score values remain in the bench JSON for future GT
extension.

### What this acceptance proves

1. **Concept-anchor injection is sufficient** — the bare proper
   noun "David Soria Parra" + the graph-walked anchor terms
   ("MCP", "Anthropic", "Model Context Protocol") together push the
   target chunk past the bge-m3 rerank threshold, where neither
   the LLM-rewritten query nor the embedding-swap alone could.
2. **Deterministic, LLM-free, sub-millisecond** — F9.2's
   `EntityAnchorExpander.expand_query` is a graph adjacency lookup
   + frequency rank, audit-replay-safe by construction.
3. **F2 pattern reused successfully** — `feedback_intent_classifier_audit_clean.md`
   established "measure with a research-driver audit before
   prescribing a fix." F9.1 followed the exact pattern: audit
   first (0/3 LLM-rewriter add-rate) → module-only PR with A/B
   arm (#546) → wiring with flag (#547) → operator measurement
   (this doc). No production code change preceded the measurement
   evidence.

### What this closure does NOT claim

- **Cross-lingual proper-noun retrieval is solved in general** —
  q15 is one fixture. The expander relies on the entity having an
  outbound graph edge to a concept; cold-start entities (newly
  ingested, no edges yet) still produce a bare-query path.
  Ingest-time anchor seeding is a v0.4.1+ candidate.
- **Mean path recall = 1.0 across step7 v4 RAG queries** — the
  current GT only annotates 4 of 16 queries. The "queries_with_expected_path = 4"
  field in the aggregate is honest about that scope. Path-GT
  expansion to q5–q14, q16 is a separate small PR.
- **F9 supersedes BL-9** — it doesn't. bge-m3 stays active. Its
  concept-side improvement (0.85 → 0.76 differential) is what gives
  the F9 anchor terms enough lift to actually clear the rerank
  threshold. Both layers are load-bearing.
- **The expander is a generic NER+anchor module** — it's wired
  for the JAMES wiki graph schema specifically (entity → relation
  → entity adjacency). Porting to a different schema (e.g.
  `core/graph` v0.5 cross-workspace federation) is a follow-up.

## Operator measurement procedure

```powershell
# .env must have all three flags active
$env:JAMES_ENABLE_ENTITY_ANCHOR = "1"
$env:JAMES_EMBEDDING_MODEL = "BAAI/bge-m3"
$env:JAMES_ENABLE_QUERY_REWRITE = "1"

# Mint an employee JWT (bypasses query.internal_rag policy gate)
$bearer = python -c "from core.auth import create_token; print(create_token('employee', api_key='2026'))"
$env:JAMES_BENCH_BEARER = $bearer

# Run step7 in retrieval mode (NOT chat passthrough)
python scripts/bench.py --suite=step7 --mode=retrieval
```

Result file lands as `reports/bench_<sha>_step7_<stamp>.json`.
Inspect `path_recall_aggregate.mean_path_recall` for top-line and
per-query `path_metrics` for granular evidence.

## Followups (not blocking F9 closure)

- **Path-GT expansion** — annotate `expected_path.nodes` on q5–q14,
  q16 in `eval/regression/step7_queries.json`. Will tighten the
  aggregate denominator from 4 to 16 and turn the bench into a
  real regression gate for graph-RAG correctness.
- **Cold-start entity handling** — first-ingest entities have no
  graph edges; the expander degrades to identity. Either seed
  anchors at ingest time (entity disambiguation pass), or detect
  cold-start at query time and fall through to bare-query.
- **`JAMES_ENABLE_ENTITY_ANCHOR` default-ON decision** — currently
  default OFF (byte-identical to pre-F9 main). Switch to default-ON
  after a v0.4.1 cycle's worth of operator-measured queries with
  zero negative deltas on the non-q15 fixtures.
- **QVT step 3 first sample** — F9 is the prototype for the
  "non-saturating quality oracle + per-PR gate" pattern that QVT
  formalizes. The acceptance procedure above is the template; the
  next cycle (`α` — non-saturating quality metric design) builds
  the oracle that runs this gate automatically.

## Related artifacts

- Memory `feedback_q15_chroma_rerank_miss.md` (F6)
- Memory `feedback_q15_chroma_embedding_root_pinned.md` (F7)
- Memory `feedback_bl9_partial_q15_query_expansion_root.md` (BL-9 partial)
- Memory `feedback_rag_cross_lingual_diagnostic.md` (the 2026-05-25 "팔란티어" original)
- Bench JSON `reports/research-runs/step7-bench-20260527_231211.json` (audit trail)
- Source PRs #545 / #546 / #547 (F9.1 / F9.2 / F9.3)
