# v0.3.x — Direction 5 Auto-routing Track

> Status: open — 2026-05-25 entry. Direction 1 (Adaptive Budgeting)
> closed (PR #461/#463/#469); D5 is the next big product cycle.
> Created: 2026-05-25
> Companion documents:
>   - `docs/handovers/v0.3.x-measurement-framework-track.md` — V3' series + 6-direction plan
>   - `reports/promo-assets/v3prime-direction1-cognitive-stages-result.md` — D1 closure + 7-tier gradient
>   - `core/reasoning/budget.py` — D1 TaskBudget module (input for D5)
>   - `docs/ARCHITECTURE.md` §Provider Contract — layer D5 sits above

## Purpose

D5 replaces the implicit *"every reasoning call uses
`JAMES_LLM_MODEL`"* assumption with an **explicit per-call backend
selection** based on task weight. Substitution-class calls
(verbatim retrieval, low-token natural stop) route to the
cheapest viable backend; heavy synthesis calls route to the
strongest available backend. The Provider Contract surface
(`core/llm_router.py` / `core/reasoning/backends/`) stays
**unchanged** — D5 is a layer *above* the contract that decides
*which* registered backend gets the call.

This is a **product cycle**, not a research cycle. Per
[Build-don't-broadcast principle](../../docs/PLATFORM_READINESS.md)
the work ships as ordinary PRs with STEP 7 bench numbers,
no LinkedIn/X tagging, no joint-paper coupling. Single Ali
design-preview DM at PR #1 merge (Provider Contract layer
courtesy), nothing else.

## Why now

| Reason | Note |
|---|---|
| D1 (`budget.py`) shipped | Provides the per-call task-weight signal D5 routes on. 7-tier natural-stop gradient (62→1681 tokens) gives D5 a measured cost axis to route against, not a guess. |
| v0.4 GOVERNANCE/EVENT track alignment | Routing decisions become audit events — D5 ships them into the same `audit_log` sink D1 budget decisions use. v0.4 picks the rows up natively without re-plumbing. |
| v0.5 Domain Pilot value proposition | Per-domain-pack model routing is the customer-onboarding pitch. D5 is the seed; v0.5 generalizes per-pack. |
| Local-first cost story | Substitution=local-small + synthesis=cloud/large is the bilingual story of *"sovereign-Ollama for retrieval, escalate only when the task earns it"*. The cost delta is concrete (e4b 62 tokens vs heavy synthesis 1681 tokens = 27× — multiply by per-1k-token pricing on the cloud side). |
| Cross-lingual RAG option 3 bundle | While the retrieval path is being touched for routing, wiki entity `aliases:` frontmatter + `entity_extract` alias resolution land in the same cycle. See [Cross-lingual section](#cross-lingual-rag-option-3-bundled) below. |

## Scope

### In

- `core/reasoning/router.py` — new module. `Router.select_backend(stage, prompt, context, budget_signal) → backend_id`
- Wiring on the 4 cognitive stages (`query_rewriter`, `planner`, `reflect`, `verify`) + `synth` to consult the router
- Flag: `JAMES_AUTO_ROUTER` — default **OFF** (opt-in, behavior-preserving). Mirrors the D1 `JAMES_ADAPTIVE_BUDGET` flag pattern
- Routing policy v1 — heuristic table keyed on `(budget_tier, task_type)`
- Backend registry: extend `core/llm_router.py` to expose the list of *available* registered backends + their declared capability tags (e.g. `tier=small|medium|large`, `provider=local|sovereign|cloud`)
- Audit row: per-routing-decision event in `audit_log` (endpoint `reason:route`)
- STEP 7 bench in PR body (CLAUDE.md rule #2) at every wiring PR
- Test contract — 1 file pinning router decisions against the 7-tier gradient inputs from D1

### Out (deferred)

- Cost-based scoring (token price × latency) — v1 is capability-tier heuristic only. v2 layer if needed
- Multi-objective routing (latency vs cost vs quality) — single-objective v1
- Backend health-check / circuit-breaker — relies on existing Provider Contract failure modes
- D6(I) joint-paper coupling — D5 is product, not research. Findings are private until D5 ships and a separate research-cycle PR (e.g. comparative bench across stacks) revisits

## D2 absorption (task-weight metric)

D2 (Task-weight metric formalization) **does not run as an
independent 1-2 week cycle**. The router needs *some* per-prompt
weight signal richer than D1's 3-tier budget; that signal is
implemented inside D5 as `router._classify_task(prompt) → tier`,
not in a separate research module.

Inputs the heuristic can use without any new measurement:

- D1's budget signal (`assess(stage, prompt, context).cap`) — already 3-tier
- D1 sub-finding: `verify`-class stages are **high-clustering**
  (~12.5% unique across 40 baseline calls) — task-type axis on
  Mechanism 2. Encoded as `_TASK_TYPE_HINTS` keyed on stage name.
- Prompt-surface signals (no LLM call): `len(prompt)`, presence of
  imperative verbs ("Return verbatim"), structured-output markers
  (JSON schema fences), Korean/English ratio (cross-lingual hint)

If at PR #2/#3 the heuristic plateau on STEP 7 bench, *that* is
the trigger to revisit D2 as a measured metric. Until then, D5
runs without D2 paper output.

## Cross-lingual RAG option 3 bundled

PR #472 (2026-05-25) landed `_SYNONYM_MAP` keyword aliases as the
fast lever (옵션 1). The graph-level permanent fix (옵션 3) is:

1. Add `aliases: [...]` field to wiki entity frontmatter schema
2. `entity_extract` resolves an alias to the canonical entity ID
3. Graph lookup is alias-aware (so a Korean query token "팔란티어"
   matches wiki entity `palantir_technologies__pltr_`)

This lands inside the D5 cycle as one of the wiring PRs — when
the router consults the retrieval path, having canonical entity
IDs (not surface-form-dependent) makes the routing signal
cleaner. Tracked as **BL-8** in
`memory/project_state.md`.

The keyword `_SYNONYM_MAP` stays (defense in depth + zero-impact
on queries that don't need entity resolution). Option 4
(embedding swap to bge-m3) is **v0.4 backlog**, not in this cycle.

## Phase plan

### D5.0 — Design memo (this PR, docs only)
- This document
- No code changes
- No bench numbers required (rule #2 N/A — docs only)

### D5.A — Router skeleton + flag (small, code + test)
- `core/reasoning/router.py` new module:
  ```python
  class Router:
      def __init__(self, *, enabled: bool = False): ...
      def select_backend(self, stage, prompt, context, budget_signal) -> str: ...
  ```
- Default `enabled=False` (gated on `JAMES_AUTO_ROUTER=1`)
- When disabled: returns the legacy `JAMES_LLM_MODEL` env value → byte-identical to pre-D5 behavior
- `tests/test_router_skeleton.py` — flag-off invariance + flag-on stub policy
- STEP 7 bench: confirm no regression (flag default OFF, bench is sanity check)

### D5.B — Backend registry capability tags
- Extend `core/llm_router.py` (or new `core/llm_router_registry.py`) so each registered backend declares `(tier, provider)`
- Pure refactor — no behavior change. Existing `JAMES_LLM_MODEL` still wins when flag OFF
- 1 unit test pinning registry shape

### D5.C — Stage wiring (4 cognitive + synth)
- 4 cognitive stages + synth call `router.select_backend(...)` when flag ON
- Per-stage routing policy default (verify → large tier on grounding-critical input; query_rewriter → small tier; etc.)
- Audit event: `reason:route` row per call with `{stage, prompt_hash[:8], budget_tier, selected_backend, reason}` payload
- STEP 7 bench in PR body — flag-OFF baseline + flag-ON measured against the 7-tier prompts. **Expected: ≥ baseline latency on substitution arm (cheaper backend), ≤ baseline on synthesis arm (better backend)**

### D5.D — Wiki entity `aliases:` + entity_extract resolution
- Wiki entity frontmatter schema: optional `aliases: [str, ...]` field
- `entity_extract` lookups now consult alias map (in-memory cache built at startup)
- Backfill the 4 high-traffic entities the PR #472 keyword map already covers (Palantir, Nvidia, Tesla, Apple) so the graph path matches the keyword path
- 1 contract test pinning alias resolution

### D5.E — Closure result doc + memory sync
- `reports/promo-assets/v3prime-direction5-router-result.md` — wiring outcomes, STEP 7 bench delta, routing-policy table
- Memory: `direction_5_auto_routing_closure.md`
- ROADMAP D5 `[ ]` → `[x]`

## STEP 7 bench plan

Baseline (flag OFF) vs measured (flag ON):

| Arm | Prompt source | Expected delta |
|---|---|---|
| Substitution | D1 7-tier tier-1 ("return verbatim") | latency ↓ (small backend) |
| Light synthesis | D1 tier-2 (e-commerce refund) | latency ≈ baseline |
| Cognitive (rewriter/planner/reflect/verify) | D1 tiers 3-6 | latency ≈ baseline on small stages; ↓ on stages routed to medium |
| Heavy synthesis | D1 tier-7 (4-stage cognitive) | latency ↑ acceptable if quality ↑ (cloud/large backend) |

Acceptance: **no regression on grounded=true rate** at any tier
(per-tier verify pass count stays at the D1 v2 closure level: 20/20
across all four cognitive stages on the 3-prompt sweep, 7-tier
natural-stop stable cross-sweep <5% per tier).

## Collaborator interaction pattern

Per memory `feedback_build_dont_broadcast.md`:

- **Ali**: single design-preview DM at PR D5.0 merge (Provider
  Contract layer courtesy — "router lands above the contract,
  no contract-shape change, mid-June Gemini backend slots in
  cleanly"). No follow-up DMs per PR.
- **Robin**: nothing — D5 is product, not measurement
- **Public**: no LinkedIn / X / blog. Closure result doc lives
  in `reports/promo-assets/` but is not promoted

This is the canonical *Build, don't broadcast* loop:
[[feedback_build_dont_broadcast]] in memory.

## CLAUDE.md rules applied

- **rule #1** (no domain features): N/A — D5 is generic infra
- **rule #2** (bench numbers on `core/{retrieval,graph,reasoning}`):
  ✅ STEP 7 bench in every D5.A~D PR body. D5.0 (this PR) docs only
- **rule #3** (self-evolution opt-in only): N/A — D5 doesn't
  touch self-evolution surface
- **rule #4** (architecture changes): D5.A introduces new module
  `core/reasoning/router.py` — qualifies as architecture surface
  change. PR carries `architecture` label. `docs/ARCHITECTURE.md`
  amended in same PR
- **rule #5** (module size ≤ 20 KB): `router.py` budget set at
  ~10 KB target. Split if approaches limit

## Done when

| Criterion | How verified |
|---|---|
| Flag-OFF default is byte-identical to pre-D5 main | `tests/test_router_skeleton.py::test_flag_off_uses_legacy_model` + STEP 7 bench delta ≤ ±5% |
| Flag-ON routes substitution → smaller backend, heavy synth → larger backend | `test_router_skeleton.py::test_tier_routing_policy` + STEP 7 bench substitution arm latency ↓ |
| Cross-lingual alias path resolves in graph lookup | `tests/test_entity_alias_resolution.py` + manual repro of 2026-05-25 팔란티어 case → rerank top = `palantir_technologies__pltr_.md` |
| Audit row written per routing decision | DB query: `SELECT COUNT(*) FROM audit_log WHERE endpoint='reason:route'` increments per chat turn |
| ROADMAP D5 [x] + closure result doc + memory sync | `direction_5_auto_routing_closure.md` exists; ROADMAP D5 checked |

## Out of scope (deferred to v0.4)

- Cost-based scoring beyond capability tier
- Multi-objective optimization
- Per-domain-pack routing policy (v0.5 Domain Pilot scope)
- Backend health probing
- Embedding model swap to bge-m3 / multilingual-e5-large (BL-9)

## Quick links

- `core/reasoning/budget.py` — D1 TaskBudget (D5 input)
- `core/llm_router.py` — Provider Contract layer (D5 sits above)
- `reports/promo-assets/v3prime-direction1-cognitive-stages-result.md` — 7-tier gradient ground truth
- `docs/PLATFORM_READINESS.md` — Build-don't-broadcast principle
- Memory: `direction_1_adaptive_budget_closure`, `feedback_build_dont_broadcast`, `feedback_rag_cross_lingual_diagnostic`, `project_state`
