# Direction α — Hybrid Cloud Tier (design memo)

> Status: **DESIGN / pre-implementation** (2026-06-03). This is the
> thinking doc that precedes any `ARCHITECTURE.md` change. A new trust
> zone (cloud egress) + new module (abstraction layer) will each need a
> separate PR to `docs/ARCHITECTURE.md` with the `architecture` label
> per CLAUDE.md rule #4 before code lands.
>
> Mother-platform work (not a domain feature). No vertical coupling.

## 0. Why this track exists

The α-measurement cycle (α-6/7/8) established, by direct measurement,
that JAMES's **answer-quality ceiling is the local model's reasoning
capability**, not retrieval and not ontology:

- Retrieval path-finding is strong and tier-invariant (S4 finding —
  citation/path survives context reshape, ±0.013 across a 27× model
  size range). **The evidence-selection stage is already solved locally.**
- α-7 (graph top-K) rejected; α-8 (typed filter) landed ⭐ operational
  only (n=3 paired collapsed n=1's apparent ⭐⭐). Neither moved the
  graded-answer ceiling, because the ceiling is **reasoning**, not
  context assembly.
- Local `gemma4 e4b` graded-answer on multihop ≈ 0.33; a frontier cloud
  model is ≈ 0.7–0.8 on comparable tasks. That gap is the product gap.

So: keep everything JAMES is already good at **local**, and route **only
the reasoning step** to a stronger model **when it's actually needed** —
without surrendering the data-sovereignty that is JAMES's reason to exist.

## 1. The routed pipeline

```
[LOCAL] query
  → intent classification + query/dialogue normalization      (IntentClassifier — exists)
  → evidence retrieval + candidate selection                  (hybrid search + graph path — exists, proven)
  → ── reasoning has NOT started yet; everything above is local & sufficient ──
  ↓
[ROUTER] reasoning-difficulty assessment                      (extends D5 AUTO_ROUTER)
  ├─ local-sufficient  → local reasoning → answer
  └─ high-difficulty / cloud-needed:
        ① abstract the selected evidence   (deterministic mask)   ← NEW: abstraction layer
        ② cloud LLM reasons over the abstracted payload           ← NEW: cloud backend tier
        ③ de-abstract the result           (deterministic unmask) ← NEW: abstraction layer (reverse)
        ④ local result analysis + answer assembly + audit record
```

The key architectural claim: **retrieval/selection is local-solved;
only reasoning is routed; cloud never sees un-abstracted sensitive
content.** Abstraction is applied at the egress boundary and reversed at
ingress, deterministically, with a local-only mapping table.

## 2. Component map — what exists vs what is new

| Stage | Mechanism | Status |
|---|---|---|
| Intent classification / normalization | `IntentClassifier` | exists (F2 audit 100%) |
| Evidence retrieval + path selection | hybrid (vector 0.6 / BM25 0.2 / kw 0.1 / name 0.1) + graph | exists, proven (S4) |
| Model routing | `AUTO_ROUTER` (D5), 3-tier capability | exists — **local tiers only today** |
| Typed entity substrate | α-8 ontology (9 types, 22 relations) | exists |
| Sensitivity tagging | chunk `sensitivity` metadata | exists |
| Audit / replay | `audit_log`, T7 supersede + replay | exists |
| Policy gate | `PolicyEngine` + ABAC | exists |
| **Reasoning-difficulty scorer** | router signal pre-reasoning | **NEW (hard problem ①)** |
| **Abstraction / de-abstraction layer** | deterministic mask/unmask over typed graph | **NEW (hard problem ②)** |
| **Cloud backend tier** | egress-gated remote LLM backend | **NEW (plumbing)** |

Roughly half the pipeline is reuse. The two NEW pieces that carry the
research risk are the difficulty scorer and the abstraction layer; the
cloud backend itself is mostly plumbing on top of the existing router.

## 3. The abstraction layer (core of the security model)

Concept: replace sensitive entities in the selected evidence with typed,
deterministic placeholders before egress; reverse the mapping on the
cloud's reply.

```
"김철수는 삼성전자의 CEO다 (연봉 30억)"
   → "PERSON_1 is the CEO of ORG_1 (QUANTITY_1)"      [egress to cloud]
   cloud reasons structurally over placeholders
   reply: "PERSON_1 reports to no one; QUANTITY_1 is above peer median"
   → de-abstract via local map → "김철수 ... 30억 ..."  [local result]
```

Why JAMES is well-positioned: the **typed graph IS the abstraction map.**
Same entity → same placeholder across all selected files; types,
relations, and cross-references are preserved by construction. The
`sensitivity` tag decides *what* gets masked. De-abstraction is a
deterministic lookup. Every transform is written to the audit log, so
"what left the machine" is provable (replay identity extends to egress).

### 3.1 The hard limit — privacy ↔ reasoning-utility tension

Deterministic masking preserves **structural / relational** reasoning
(who-reports-to-whom, causal chains, comparison-by-magnitude) but breaks
reasoning that needs the entity's **real-world semantics** (e.g. "is this
drug interaction dangerous?" with `DRUG_1`). Therefore **you cannot mask
everything.** A policy must split:

- *maskable* — entity identity is irrelevant to the reasoning (structure
  only) → mask.
- *must-pass-through* — reasoning needs the real semantics → either pass
  through (accept the data exposure) or **keep local** (don't route to
  cloud at all).

This split is the second hard problem (§5.2). It is where the
sensitivity tags + ontology types + PolicyEngine combine.

### 3.2 PoC status (2026-06-03)

`scripts/research/abstraction_layer_poc.py` — deterministic mask/unmask
proof-of-concept, validated locally (no cloud, no cost). It de-risks the
abstraction-layer security claim before any cloud-egress code:

- ✅ closed-world reasoning survives masking — masked org-chart
  ("PERSON_3 → PERSON_2 → PERSON_1") reasons correctly and restores to
  real names (same entity → same placeholder = structure preserved)
- ✅ hallucinated placeholder (not in map) is flagged, never restored
- ✅ open-world sensitive entity routed to KEEP_LOCAL (no egress)
- ✅ deterministic; substring-safe (`김철` ⊂ `김철수`); Korean-particle-safe

**End-to-end against a REAL cloud model (2026-06-03):**
`scripts/research/abstraction_e2e_claude.py` runs the full loop —
mask → Claude (Max-plan, headless `claude -p`) reasons over placeholders
→ unmask — and it works: masked org-chart in, `"PERSON_1입니다"` reasoned
by Claude over placeholders, de-abstracted to `"김철수입니다"`, real names
never egressed, no hallucinated placeholder. Proves the closed-world
masking claim holds against a real frontier reasoner, not just a
simulated reply. **Max plan is the free research stand-in for the cloud
tier** (bounded own-research use; a production tier still needs the
Anthropic API + trust-zone PR — Max is not a production backend).
Invocation gotchas recorded: run from a neutral cwd (else it loads the
project CLAUDE.md and goes coding-agent mode), pass the prompt via
**stdin** not argv (Windows mangles multi-line/UTF-8 argv).

**Still open (the real R&D, §4):** query-conditioned semantic-dependence
classifier (PoC approximates with explicit open-world sets), difficulty
routing, real cloud egress (needs the ARCHITECTURE trust-zone PR), and
wiring to live graph entities + `sensitivity` metadata (PoC uses
synthetic entity dicts). Promotion to a `core/` module is gated on the
`architecture`-labelled PR per CLAUDE.md rule #4.

## 4. The two hard problems (the actual R&D)

Everything else is plumbing. These two are the track:

### 4.1 Problem ① — reasoning-difficulty routing (chicken-and-egg)

Decide "local-sufficient vs cloud-needed" **before** the reasoning runs.
Four candidate strategies, each with a trap:

| Strategy | Mechanism | Trap |
|---|---|---|
| A. Pre-signals | predict from cheap retrieval-stage signals (hop count, evidence conflict, intent type) | proxies — a short query can be hard, a multi-hop one trivial |
| B. Local quick-pass + confidence | local answers, escalate if unsure | **local self-confidence is most miscalibrated on hard cases** (confidently wrong — cf. e4b thinking-trace) |
| C. Local answer + grounding/verify check | check the answer is grounded in the evidence + self-consistent; escalate on fail | verifier quality is the ceiling |
| D. Learned router | train (query features → was-cloud-needed); the audit log is the training data | cold-start; needs cloud ground-truth accumulated |

**The crux insight:** difficulty-estimation difficulty correlates with
the difficulty itself — the cases where local confidence is most wrong
are exactly the ones to escalate. **So never trust the model's
self-assessment.** Use a grounding check (answer ↔ retrieved-evidence
overlap, which JAMES already does for citation/abstention) as the
backstop — it checks the answer without trusting the model.

Also: "cloud needed" has two causes — **(a) local can't (capability,
the real target)** vs **(b) local is unsure but might be right (escalating
wastes cost)**. Grounding + consistency separates them: ungrounded /
inconsistent ⇒ (a) ⇒ escalate; grounded + consistent but low
self-confidence ⇒ fine locally.

**Recommended design — layered, JAMES-native:**

```
1. cheap pre-filter (A)          → obvious cases routed at zero cost
   (single coherent source lookup → local; obvious multi-hop+conflict → cloud)
2. local quick-pass + grounding/consistency check (B+C)  → ambiguous middle only
   (abstain / low-grounding / inconsistent → escalate)
3. threshold = OPERATOR DIAL     → not a hardcoded constant
   (finance deployment escalates readily; casual one is conservative)
4. audit log accrual → learned calibration (D)  → 1+2 thresholds self-tune over time
```

Threshold is an operator dial because the escalate-readiness is
deployment- and sensitivity-class-dependent — this is the same
"operator-choice tier" knob as the framing (§6).

**Closure experiment** (data, not speculation): on the MultiHop fixture
+ one cloud model — (1) identify cases where local graded-answer is low,
(2) measure which pre-signals (hop / conflict / intent) correlate with
those, (3) confirm the grounding check actually separates (a) from (b).
That defines which signals predict "cloud-needed."

**First run (2026-06-03) — premise NOT yet confirmed.**
`scripts/research/local_vs_cloud_multihop.py` v2 (reasoning-isolated: same
full gold evidence to both, blinded LLM-judge) gave **local gemma3:4b 9/9
= cloud Claude 9/9 — zero quality gap** on clean-evidence multihop. The v1
"cloud loses" was a `num_ctx=2048` truncation artifact + gold_signal-
fragment metric artifact. Implication: the "local reasoning ceiling" that
motivates this whole track may be a **retrieval + metric artifact**, not a
reasoning limit — see `memory/project_direction_alpha_local_vs_cloud_
quality_thread`. Caveats keep it "unproven" not "refuted": lenient judge
(contradictory answers both CORRECT), gold-evidence ≠ real noisy
retrieval, easy first-N cases, small n, gemma3:4b not e4b.

**Sequencing decision (2026-06-03):** do NOT chase the differentiation
test standalone now. **Build the cloud tier first; run local-vs-cloud
quality differentiation as an extension of the α-8 test line, integrated
and progressive, alongside the build.** Cloud-tier value must be *proven*
(harder cases / noisy retrieval / null-query hallucination / strict judge
/ real pipeline), but proving it is not a blocking prerequisite to the
build — they proceed together.

**Residual risks:** pre-signals (A) are forever proxies — goal is a
cost/quality Pareto improvement, not perfect routing; step 2 double-pays
(local + cloud) on escalated cases, so the cheap pre-filter must catch
most; learned calibration (D) only starts once cloud ground-truth
accumulates.

### 4.2 Problem ② — egress masking policy

Three-way outcome **per entity** (not per document — the typed graph
gives entity-level granularity, so one doc can be partially masked):

- **mask** — replace with a typed placeholder (`PERSON_1`, `ORG_1`)
- **pass-through** — send the real value
- **keep-local** — don't route to cloud at all

Decided by a 2×2 of **sensitivity** (the `sensitivity` tag) × **semantic
dependence** (does the reasoning need the entity's real-world meaning):

| | not sensitive | sensitive |
|---|---|---|
| reasoning needs only structure/relations | mask or pass (don't care) | **MASK** ← sweet spot |
| reasoning needs real-world meaning | pass-through | **KEEP-LOCAL** ← hard corner |

**The test for "semantic dependence" — closed-world vs open-world:**

- **closed-world** — the answer is derivable *from the provided
  documents'* structure. The entity is just a label; its identity is
  irrelevant. Masking preserves everything needed (relationships survive
  consistent masking), so the cloud reasons correctly over placeholders
  and the result de-abstracts cleanly. → **maskable.**
  *(Org-chart example: "PERSON_C → PERSON_B → PERSON_A (top)" reasons
  correctly with all three masked, because only the reporting structure
  was needed.)*
- **open-world** — the answer needs the cloud model's outside-world
  knowledge *about that specific entity* (e.g. "is warfarin+aspirin
  dangerous?" — masking the drug names destroys exactly what's needed).
  → **not maskable** → keep-local or explicit consent.

**Why this fits JAMES:** JAMES is a grounded/retrieval system, so most of
its reasoning is closed-world by nature (answer from the provided
evidence). That means **most sensitive content is maskable** — the
open-world minority is where keep-local / consent applies. The
abstraction layer therefore covers the common case, not the edge case.

**Inputs JAMES already has:** `sensitivity` tag (is it sensitive),
ontology type (what kind of entity), typed graph (entity-level mask
boundaries + same-entity→same-placeholder consistency), audit log
(provable record of what left the machine).

**Hard-corner escape hatches** (sensitive + open-world): (1) keep-local —
accept lower quality, keep sovereignty (default); (2) explicit
operator/user consent via `PolicyEngine` to expose specific entities;
(3) expose the minimum subset only.

**Safety:** a cloud-introduced placeholder not in the local map (a
hallucinated entity) must be **flagged, never silently de-abstracted.**

## 5. Cost model (why routed, not always-cloud)

Cloud is the quality lever but also the cost + sovereignty cost. Routing
only the hard reasoning steps means: most queries stay fully local (zero
marginal cost, zero egress); only the genuinely hard minority pay cloud
cost. The router's threshold is therefore also the **cost dial**. A clean
Pareto measurement (local-only vs routed vs always-cloud, on quality and
cost) is the eventual acceptance evidence.

## 6. Framing

**"Data-sovereign with operator-choice tier."** Not "local-first" (that
undersold the quality story) and not "cloud RAG" (that surrenders the
moat). The operator chooses, per deployment / per sensitivity class,
whether a given query class may egress — and *when it does, only
abstracted, audited content leaves.* The sovereignty guarantee is the
abstraction boundary + the audit record, not the absence of cloud.

## 7. Non-goals / guardrails

- ❌ No always-on cloud. Local stays the default; cloud is opt-in and
  per-query gated.
- ❌ No un-abstracted egress of `sensitivity`-tagged content. Ever.
- ❌ No domain coupling (mother-platform rule until v1.0).
- ❌ No bypass of `PolicyEngine` for the egress decision.
- ❌ Cloud-introduced entities not in the abstraction map (hallucinated
  placeholders) must be flagged, never silently de-abstracted.

## 8. Open questions → next steps

1. **Difficulty scorer** (§4.1) — deep-dive analysis: which signals,
   how combined, threshold calibration, measurement plan.
2. **Egress policy** (§4.2) — deep-dive analysis: the mask /
   pass-through / keep-local decision table over sensitivity × type ×
   semantic-dependence.
3. Cloud backend tier registration into `AUTO_ROUTER` (plumbing; after
   1+2 shape the contract).
4. Pareto measurement design (local / routed / always-cloud).
5. `ARCHITECTURE.md` PRs: (a) cloud-egress trust zone, (b) abstraction
   module — each `architecture`-labelled.

## 9. 한국어 요약

로컬 품질 천장 = 로컬 모델의 **추론 능력** (검색·온톨로지 아님, α-cycle 로 실측). 그래서 JAMES 가 이미 잘하는 것(분류·검색·선별)은 전부 **로컬 유지**, **추론 단계만** 필요할 때 강한 클라우드 모델로 라우팅. 단 클라우드는 **추상화된 내용만** 보고, egress 경계에서 결정론적 마스킹 + 복귀 시 역마스킹, 매핑 테이블은 로컬 전용, 모든 변환은 audit 기록. **typed graph 가 곧 추상화 map** (같은 엔티티→같은 placeholder, 타입/관계/cross-ref 보존), sensitivity 태그가 "무엇을 가릴지" 결정. 진짜 R&D 난제 2개: ① 추론-전 난이도 판정(닭-달걀), ② "마스킹 가능 vs 통과 필요 vs 로컬 고정" egress 정책. 나머지는 배관. Framing = "data-sovereign + operator-choice tier". v0.5 도메인 pilot **앞에** 와야 품질 천장을 먼저 풀고 도메인 검증이 깨끗함.
