# Design Memo — α-7 Graph Top-K Filtering

> **Status**: design (pre-PR). Successor to α-6 cycle (Phase 1+2 closed,
> Phase 3a in flight as of 2026-06-01 PM).
> **Trigger**: α-6 Phase 1 §3 finding — graph layer is the net regressor
> on `graded` axis at M_M production tier (Δ -0.053 vs C_rag-basic).
> 41-161 entities surface per query at the C_rag-graph cell, swamping
> the LLM context window with low-signal nodes.
> **Cycle position**: first cycle that ships a JAMES code change against
> α-5+α-6 measurement debt. Cumulative count before this PR: **0 lines**.
> Per 2026-06-01 strategy handover §1.2: "highest-ROI fix, ~1 day code."
> **작성일**: 2026-06-01 PM (post α-6 Phase 3a 12b lands, pre 27b lands)

---

## 0. TL;DR

The graph layer (`expand_dynamic` in `core/graph_engine.py:314-358`)
surfaces entities by DFS with score-threshold halting. There is no
top-K limit; the surface count grows with workspace density. At the
931-entity MultiHop-RAG workspace, this yields 41-161 entities per
query — enough to push the C_rag-graph cell into graded regression at
M_M (-0.053) per Phase 1.

α-7 caps the surface with a small **post-DFS top-K filter** in a new
module `core/graph_topk.py` and **tightens** `DFS_SCORE_THRESHOLD`
from 0.05 to 0.08. Minimal mechanism change; no migration; no schema
edit; preserves existing test surface. The PR ships against a
re-baseline (multihop_rag, ~107 min) and is judged by 5-axis Quality
Delta Card vs the post-PR `baseline_<sha>_rescored.json`.

---

## 1. Problem statement

### 1.1 What α-6 measured

| Cell at M_M (gemma4:e4b) | path | graded | abst_f1 |
|---|---:|---:|---:|
| C_minus (pure LLM) | 0.000 | 0.347 | 0.558 |
| C_rag-basic (+ S1 RAG) | 0.000 | 0.327 | 0.421 |
| **C_rag-graph (+ S2+S3+S4)** | 0.411 | **0.273** | 0.462 |
| C_rag-full (+ S5+S6, = L1) | 0.419 | 0.327 | 0.591 |

The C_rag-basic → C_rag-graph step pays:
- path: +0.411 (S4 citation contribution)
- graded: **-0.054** (S2 graph regression)
- abst_f1: +0.041 (partial recovery)
- latency: +16.1s

S5+S6 (next step) recovers graded back to baseline (+0.054), but at
8.4× total latency. If S2 didn't regress graded in the first place,
the latency tax could be paid only for the abst_f1 gain (+0.129 of
S5+S6 contribution), not also for graded restoration.

### 1.2 Mechanism (`feedback_oracle_phrase_artifacts` 4-step applied)

```
core/graph_engine.py:314 — expand_dynamic(entity_ids, source_type_filter):
    DFS visits entities at depth d=0..MAX_DEPTH (=4),
    halts on ACT score < DFS_SCORE_THRESHOLD (=0.05),
    filters by relation confidence > CONFIDENCE_THRESHOLD (=0.6),
    appends every passing entity to `entities`.

NO top-K. NO frequency penalty. NO context-window awareness.
```

The 41-161 surface count is the natural growth of DFS at depth 4 on
a 931-entity workspace where most entities pass the 0.05 score
threshold.

### 1.3 Honest framing tier

⭐ **operational** (per `memory/feedback_finding_size_honest_framing`).
"Top-K filter on graph traversal" = a standard GraphRAG technique
(MS GraphRAG, Neo4j LLM KG Builder both do variants). Adopting it
is a *bug fix*, not a *finding*. The PR's value is in the *measured
delta* (Quality Delta Card numbers), not in the *mechanism*.

---

## 2. Scope

### 2.1 In scope (this PR / cycle)

1. **New module** `core/graph_topk.py` (~2-3 KB target):
   - `top_k_filter(entities, paths, k=10) → (filtered_entities, filtered_paths)`
   - Ranks by existing `_dfs_score` field (populated by `expand_dynamic`)
   - Drops entities below rank K; drops corresponding paths
   - No new dependency
2. **Wire** `expand_dynamic` to call the filter before return.
3. **Tighten** `DFS_SCORE_THRESHOLD` 0.05 → 0.08 in `core/graph_engine.py`.
4. **Re-baseline**: `scripts/qvt_capture_baseline.py` against multihop_rag
   balanced-100, write `eval/qvt/baseline_<sha>_rescored.json`.
5. **5-axis Quality Delta Card** comparing C_rag-graph cells before
   vs after (against the new baseline).
6. **ARCHITECTURE.md** — register `core/graph_topk.py` under §graph
   subsystem.
7. **Memory** — promote `graph-layer-bug-confounds-all-graded-deltas`
   (Phase 3a 12b finding §7) from `bucket-d-measurement-debt` to
   `bucket-a-resolved`.

### 2.2 Out of scope (deferred or rejected)

- **PageRank-style centrality scoring** — over-engineering for the
  measured problem; the cheap K filter should handle 90% of overload.
- **Per-depth top-K** — more surgical but adds 15-20 lines; only
  warranted if simple top-K doesn't move graded back into noise band.
  Deferred to a sub-finding investigation if needed.
- **Frequency penalty across queries** — would require workspace-level
  state; orthogonal to per-query top-K.
- **Edge-weight aware ranking** — `compute_graph_score` already weights
  by ontology relation weight; this isn't decoupled from DFS scoring.
- **Ontology-aware filtering** — explicitly deferred to α-8 (the next
  cycle).
- **`graph_engine.py` split to satisfy 20 KB gate** — the file is
  currently 20923 bytes (just over). α-7 lives in a *new* module
  precisely to avoid forcing a split now. The split is its own
  refactor PR, deferable.

---

## 3. Implementation sketch

### 3.1 New module shape

```python
# core/graph_topk.py
"""α-7: top-K filter for graph DFS entity surface.

Bound the per-query entity surface to the K highest-ranked entities
by their existing `_dfs_score`. Drop the corresponding reasoning
paths whose tail entities are filtered out.
"""

from typing import Dict, List, Tuple

# Operator-tunable; production default chosen at the empirical
# inflection point where 95th percentile of MultiHop-RAG queries no
# longer overload the context window. Justify with re-baseline data.
DEFAULT_TOP_K = 10

def top_k_filter(
    entities: List[Dict],
    paths: List[str],
    k: int = DEFAULT_TOP_K,
) -> Tuple[List[Dict], List[str]]:
    """Return at most K entities (by _dfs_score) and the subset of
    paths whose tail entity survives."""
    if len(entities) <= k:
        return entities, paths
    # Sort descending by _dfs_score; preserve original list order for
    # ties so the DFS visit-order tiebreak is deterministic.
    ranked = sorted(
        entities,
        key=lambda e: (-e.get("_dfs_score", 0.0), entities.index(e)),
    )
    kept = ranked[:k]
    kept_ids = {e.get("entity_id") or e.get("id") for e in kept}
    kept_paths = [p for p in paths if _path_tail_id(p) in kept_ids]
    return kept, kept_paths

def _path_tail_id(path_str: str) -> str:
    """Extract the tail entity id from a reasoning path string."""
    # Path format: 'e_concept_abc12345 → relation → e_org_def56789'
    # The trailing 'e_...' token is the tail.
    tokens = path_str.split("→")
    if not tokens:
        return ""
    return tokens[-1].strip()
```

### 3.2 Wire point

```python
# core/graph_engine.py:expand_dynamic — at return:
-    return entities, paths
+    from core.graph_topk import top_k_filter
+    return top_k_filter(entities, paths)
```

### 3.3 Threshold tighten

```python
# core/graph_engine.py:38
-DFS_SCORE_THRESHOLD = 0.05
+DFS_SCORE_THRESHOLD = 0.08  # α-7: tighter ACT halting; complements top-K
```

---

## 4. Success criteria (Quality Delta Card pre-commit)

PR description must include the 5-axis card. Build-before-claim:

| Axis | Pre-α-7 baseline (M_M, n=1) | Post-α-7 (M_M, n=1) | Δ | Acceptance |
|---|---:|---:|---:|---|
| path_coverage | 0.404 | TBD | TBD | flat (S4 citation is separate; α-7 must not break it) |
| graded_answer | 0.343 | TBD | TBD | **+0.030 or better** (recover the C_rag-graph regression of -0.054) |
| abstention_f1 | 0.609 | TBD | TBD | flat ±0.05 (don't degrade abstention) |
| token_cost | 1150 | TBD | TBD | ≤ -100 (smaller surface → fewer tokens injected) |
| latency_cost | 64s | TBD | TBD | ≤ -5s (less LLM context to process) |

If `graded_answer Δ ≥ +0.030` and other axes survive the acceptance
band, the PR lands as `feat(α-7): graph top-K`. Otherwise the PR is
re-shaped (per-depth top-K? different K? threshold revert?) before
landing.

⚠️ **Build-before-claim discipline** — the table above is empty until
the bench runs. The merge PR must paste the filled card, not a
prediction.

---

## 5. Risks + mitigations

| Risk | Probability | Mitigation |
|---|---|---|
| Top-K=10 cuts queries that legitimately need 20+ entities (e.g. comparison queries) | medium | per-question-type cross-tab in the Quality Delta Card; per-type acceptance band, not aggregate only |
| Tightening threshold 0.05→0.08 breaks single-hop queries where only one entity has score 0.05-0.08 | low-medium | preserve C_rag-basic baseline (no graph) as fallback per query; verify in cross-tab |
| `_path_tail_id` parser breaks on path format variants | medium (depends on path format stability) | unit test on representative paths from existing audit log; round-trip parser sanity |
| Filter applied at wrong level (per-DFS-call vs per-query) and causes deduplication weirdness | medium | follow the existing `expand_dynamic` return contract — one filter call per outer query, not per recursion |
| Threshold change interacts non-linearly with K (both contribute to surface size) | medium | A/B: PR-A K-only, PR-B threshold-only, then combined — if combined is better than max(A, B) the interaction is positive |
| `graph_engine.py` split blocked because dependents add to it | low | new module avoids touching graph_engine; size delta from this PR ≈ +3 KB net (new module) and ≈ +0 in graph_engine |

---

## 6. Cycle dependencies

### 6.1 Predecessor

- α-6 Phase 3a closure PR — must land first to set the canonical
  baseline against which α-7 measures.
- The α-6 Phase 1 §3 graph regression finding — recorded; quoted in
  this memo §1.

### 6.2 Successor

- **α-8 ontology typed-filter** — α-7 ships heuristic K bounding;
  α-8 measures whether a typed schema does better than heuristic K
  (A/B isolation: Δ = ontology_cell − topK_cell).
- **T3 Evidence Aging** — orthogonal to graph entity surfacing; can
  run in parallel after α-7 lands.

### 6.3 Sequencing rule

Graph-touching cycles serial (α-6 → α-7 → α-8). Confidence/time
cycles parallel (T3 with α-7 or α-8). Per 2026-06-01 strategy
handover §4 user decision.

---

## 7. References

- α-6 Phase 1 analysis (M_M C_rag-graph regression): `reports/research-runs/alpha-6-phase-1-analysis-20260601.md`
- α-6 Phase 3a 12b analysis (graph-bug-confound finding): `reports/research-runs/alpha-6-phase-3a-gemma3-12b-analysis-20260601.md` §7
- Strategy handover (priority of graph fix): 2026-06-01 PM user message §1.2 + §3 + §7
- Honest framing rule: `memory/feedback_finding_size_honest_framing.md`
- 4-step rule: `memory/feedback_oracle_phrase_artifacts.md`
- Existing graph DFS: `core/graph_engine.py:314-358`
- Existing ontology (relation weights): `core/ontology.py`
- Workspace baseline procedure: `scripts/qvt_capture_baseline.py`
- 5-axis Pareto verdict: `scripts/qvt_ablation_matrix.py::_classify_five_axis_delta`
- ARCHITECTURE.md graph subsystem section (to extend)

---

## 8. Open decisions

| # | Decision | Recommendation | Owner |
|---|---|---|---|
| 1 | DEFAULT_TOP_K value | 10 (post-α-6 inflection candidate; revisable per measurement) | operator |
| 2 | DFS_SCORE_THRESHOLD value | 0.08 (small step from 0.05; safe-side) | operator |
| 3 | Per-depth top-K added in this PR or deferred? | deferred (only if simple K insufficient) | operator |
| 4 | A/B split (K-only / threshold-only / combined) — one PR or two? | one PR with combined; A/B in cross-tab not branches | recommend single PR |
| 5 | Sub-finding investigation gate (when do we escalate) | if `graded Δ < +0.015` after combined | recommend pre-commit |
