# Sibyl v0.7 Native Memory Core Spec

- Status: implementation in progress after cross-model spec review
- Target release: v0.7
- Tracking epic: `epic_564b41ff89d6`
- Primary outcome: make `remember -> recall/context -> reflect` run on native SurrealDB primitives
  with measurable quality and policy safety.

This spec turns the post-v0.6.0 roadmap into an executable plan. It assumes SurrealDB is the default
runtime, legacy PostgreSQL and FalkorDB active surfaces are gone, and Graphiti-on-Surreal is
transition scaffolding rather than the northstar.

---

## 1. Goal

Ship the first pure-Surreal memory core:

- raw memory remains the source of truth for capture and provenance
- context packs become the user-facing retrieval product
- reflection creates native decisions, procedures, tasks, artifacts, and relationships
- policy is enforced before memory is selected, rendered, reflected, or shared
- Graphiti is removed from the default `remember`, `recall`, `context`, and `reflect` loops after
  native behavior is measurably better

The release is not "Graphiti rewritten in SurrealQL." It is a Sibyl-native memory system optimized
for agents that need precise, scoped, source-grounded context.

## 2. Why Now

v0.6.0 proved the Surreal default runtime and opened Phase 3. The next commits landed:

- SurrealDB server image pinned to `surrealdb/surrealdb:v3.0.5`
- Python SDK locked to `surrealdb==2.0.0`
- raw `remember` source capture for CLI and MCP
- raw memory latency and keyed-scope write guards
- direct SurrealQL spike covering raw memory, entity, episode, relationship, lexical search, vector
  search, graph traversal, and context-pack rendering

The system is ready for the larger move, but only if we install a quality scoreboard before
replacing more behavior.

## 3. Non-Goals

v0.7 should not attempt:

- full admin UI for policy or memory-space management
- arbitrary cross-org sharing
- bulk mailbox or archive ingest as the main product milestone
- deleting Graphiti before native retrieval, reflection, temporal behavior, and source-grounded
  summary behavior are measured
- replacing every MCP/API/CLI surface in one pass
- building custom role systems beyond the minimum memory-policy contract
- shipping graph-guided `synthesize`; that remains post-v0.7

## 4. Success Criteria

v0.7 is done when all of these are true:

- A seeded eval harness blocks regressions in source grounding, permission safety, latency, token
  budget, and task usefulness.
- A minimal memory policy primitive lands before native retrieval or native writes depend on scoped
  data. Wave 1 makes read checks concrete and adds default-deny write, share, and reflect decisions
  for scope crossings until Wave 5 expands them.
- A native retrieval path can build context packs from raw lexical search, graph full-text search,
  vector search, and graph neighborhood expansion.
- At least one production write path creates native entity, episode, and relationship records
  without Graphiti performing the write.
- Reflection can promote a raw session capture into native decisions, procedures, artifacts, tasks,
  and relationships with source links.
- Memory policy is centralized and used by recall, context, wake, remember, reflection, CLI
  commands, MCP tools, and API routes.
- Graphiti has a concrete removal inventory with default-loop call sites classified by behavior and
  either replaced, gated, or explicitly deferred.
- Pre-v0.7 Graphiti-written `Episodic` and `Entity` records that can be projected into explicit
  scope and source metadata remain queryable through native retrieval after the default-loop flip.
  Unprojectable records are inventoried and excluded rather than default-allowed.

## 5. Product Stories

### Coding Agent Handoff

An agent asks for project context before changing code. Sibyl returns active work, current
decisions, changed files, risks, and exact test evidence with source IDs. Private memories from
another principal do not appear. A delegated agent acting for Bliss may see her project-scoped
memory, but not her private memory unless the delegation explicitly grants the private scope.

### Personal Memory Recall

An agent asks about a home or preference routine. Sibyl returns only memories visible to that
principal and memory space. Project or team work memories do not leak into the personal pack.

### Session Reflection

After a development session, Sibyl preserves the raw transcript, extracts durable decisions and
procedures, links every derived record to the raw source, and marks candidates that need review.

### Agent Diary

A named agent can write private diary notes for recurring project gotchas. Those notes appear in
`wake` or `recall` only when principal, agent identity, project, and memory-space scope allow it.

## 6. Target Architecture

### 6.1 Native Memory Pipeline

1. Capture source material into `RawMemory`.
2. Authorize the requested memory action, scope, and principal through the memory policy primitive.
3. Search raw sources and native graph records with scoped filters.
4. Fuse lexical, vector, graph, recency, and task-state signals.
5. Render a context pack with source, reason, visibility, freshness, and token budget metadata.
6. Reflect selected raw captures into native records.
7. Expose recall, remember, reflection, and promotion decisions for audit and tests.

### 6.2 Core Primitives

- `Principal`: discriminated union for user or agent identity. Agent principals carry both
  `principal_id` and `acting_for_user_id` / `agent_identity` so audit fields do not collapse the
  actor and the human being represented.
- `MemoryScope`: private, delegated, project, team, organization, shared, public.
- `MemorySpace`: policy boundary for scoped recall and promotion.
- `Source`: source adapter identity, source version, privacy class, and original metadata.
- `RawMemory`: verbatim capture with source ID, principal ID, scope, provenance, and capture time.
- `NativeEntity`: decision, procedure, task, artifact, claim, note, project, person, domain, or
  other typed memory object.
- `NativeEpisode`: session or event memory with source links and temporal metadata.
- `NativeRelationship`: typed edge with source links, confidence, validity, and supersession
  metadata.
- `ContextPack`: agent-facing selected memory bundle.
- `ReflectionCandidate`: extracted record awaiting policy, review, or promotion.
- `MemoryPolicyDecision`: allow/deny plus reason and scope source.

### 6.3 Surreal Tables and Relations

Existing tables stay in place for the first slice:

- `raw_captures`
- `entity`
- `episode`
- `relates_to`
- `mentions`

v0.7 may add:

- `memory_spaces`
- `sources`
- `reflection_candidates`
- `visibility_edges` or a relation equivalent for scoped sharing
- `relates_to` edges with `name = "SUPERSEDES"` for decision and fact replacement

Do not add new tables until a test needs the behavior. Prefer using existing Surreal graph tables
for the first production write adapter.

A dedicated memory audit table is deferred beyond v0.7 unless Wave 5 proves it is needed for the
central policy contract. The release still requires policy decisions to carry reason strings and
enough metadata for tests, structured logs, responses, and future audit storage.

### 6.4 Evaluation and Runtime Contracts

The v0.7 scoreboard uses concrete, testable metrics:

- Leak count is any rendered context-pack item whose source ID matches a fixture's
  `forbidden_source_ids`, or whose visible text contains a fixture's `forbidden_terms`.
- Source metadata coverage is
  `(items with non-empty source_id, visibility, reason, and freshness) / items`. Fixtures tagged
  `source-grounding` require coverage 1.0.
- Token estimates use a small estimator interface. Prefer an exact tokenizer when one is already
  available in the runtime. OpenAI-compatible budgets use `tiktoken` with the `cl100k_base` encoder
  when it is available; fall back to the evaluator's `characters / 4` estimate only when the report
  labels the method as approximate and keeps a 20% safety margin against the hard budget.
- Deterministic pass/fail results require a fixed embedder, fixed index build settings, and seeded
  fixtures. The seeded suite pins `SIBYL_EMBEDDER_MODEL`, embedding dimensions, index settings, and
  tokenizer revision in eval metadata and fails if the recorded runtime drifts. If a fixture sits
  inside a documented threshold tolerance band, revise the fixture instead of gating on it.
- Policy decisions in v0.7 surface in two places: structured server logs with `surface`,
  `principal_id`, `memory_scope`, `scope_key`, `reason`, and `action`; and per-item `policy_reason`
  metadata on rendered context packs and remember/reflect responses.
- Native vs. Graphiti retrieval is selected by `SIBYL_RETRIEVAL_MODE`: `native` is the default,
  `graphiti` is the named transitional fallback, and `compare` runs both, returns native, and logs
  diffs.
- Compare-mode diff logs are policy-safe: Graphiti results are filtered through the Wave 1 read
  policy before comparison, and diff records log source IDs, counts, and reason codes instead of raw
  memory text.

## 7. Work Plan

### Wave 0 - Spec, Tracking, and Baseline

Tracking: `epic_564b41ff89d6`

Purpose: lock the contract before implementation starts.

Tasks:

- Create this spec and link it from `SIBYL_NORTHSTAR.md`.
- Create a Sibyl epic for v0.7 and child tasks for each wave.
- Keep `SURREALDB_PHASE3_BURNDOWN.md` focused on dependency deletion, not product behavior.

Verify:

- `moon run docs:lint`
- Sibyl task list shows the v0.7 epic and first implementation task.

### Wave 1 - W2.5 Evaluation Scoreboard

Tracking: `eef209f1-59ea-4a1c-a3bc-8fd871804d9d`

Purpose: make "better context" measurable before changing retrieval.

Files:

- `packages/python/sibyl-core/src/sibyl_core/evals/context.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/evals/runtime.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/auth/memory_policy.py` [create]
- `packages/python/sibyl-core/tests/test_context_pack_evals.py` [expand]
- `packages/python/sibyl-core/tests/test_evals_runtime.py` [expand]
- `packages/python/sibyl-core/tests/test_memory_policy.py` [create]
- `benchmarks/context_pack_eval.py` [expand]
- `docs/testing/benchmark-methodology.md` [expand]

Implementation:

- Freeze context-pack fixtures into named suites: `coding-handoff`, `personal-memory`,
  `project-recall`, `delegated-recall`, `agent-diary`, `private-leak-negative`,
  `stale-decision-replacement`, and `source-grounding`.
- Require any compressed or summary text in `source-grounding` fixtures to cite raw or native source
  IDs before it can count as useful context.
- Add a pure read-side memory policy decision helper for private, project, and delegated scopes.
  API, CLI, and MCP surfaces can supply principal, agent, project, and membership context without
  owning the decision rules.
- Add `authorize_memory_write`, `authorize_memory_share`, and `authorize_memory_reflect` decisions
  with closed reason codes. Write and reflect decisions allow same-scope operations only when the
  caller supplies principal, scope, and verified membership context. Share remains deny-only until
  `memory_spaces` enables an explicit allowed case. The helpers deny scope crossings, missing scope
  keys, unverified membership, principal mismatches, missing agent identity, and disabled scopes
  with stable codes: `scope_not_enabled`, `missing_scope_key`, `unverified_membership`,
  `scope_crossing_requires_promotion`, `principal_mismatch`, and `agent_identity_required`. Wave 5
  expands allowed cases; Waves 3 and 4 must call the helpers instead of inventing local policy.
- Add hard gates for max estimated tokens, max latency, source metadata coverage, forbidden terms,
  and required scoped metadata.
- Add fixture data that includes private memories from another principal and confirms they are
  omitted.
- Make the token estimator explicit in reports, including the estimator method and any approximate
  safety margin.
- Measure p95 over 20 runs on the standard CI runner image and document the CI-specific threshold in
  `docs/testing/benchmark-methodology.md`. Until that measurement lands, CI uses a 3x local latency
  threshold.
- Keep `stale-decision-replacement` green with scripted superseded metadata on raw captures in
  Wave 1. Wave 4 now exercises native `SUPERSEDES` edges in the post-reflection recall fixture;
  migrate scoreboard replacement evidence onto that path when the eval harness consumes it.
- Persist eval reports under `.moon/cache/evals` for local runs and expose a concise summary.

Acceptance:

- seeded context-pack eval pass rate is 1.0
- frozen suite membership is explicit and any new fixture is additive
- source metadata coverage is 1.0 for required cases
- forbidden private-memory fixtures produce 0 leaks
- `wake` stays under 1,200 estimated tokens
- `recall` stays under 2,000 estimated tokens unless the case opts into a higher limit
- local recall context-pack p95 stays under 1s for seeded fixtures, while CI uses the documented
  runner-specific gate
- the frozen suite can run twice locally with identical pass/fail outcomes

Verify:

- `moon run core:test`
- `moon run core:bench-context`
- `moon run core:lint core:typecheck`

### Wave 2 - W7 Native Retrieval Baseline

Tracking: `40eddb63-f40d-48be-a892-29920864a320`

Purpose: build a scoped retrieval plan that context packs can use without depending on Graphiti
hybrid search.

Files:

- `packages/python/sibyl-core/src/sibyl_core/tools/context.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/tools/search.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/retrieval/hybrid.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/retrieval/fusion.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/graph/search_interface.py` [expand]
- `packages/python/sibyl-core/tests/test_context_pack.py` [expand]
- `packages/python/sibyl-core/tests/test_context_pack_evals.py` [expand]
- `packages/python/sibyl-core/tests/graph/surreal/test_search_interface.py` [expand]

Implementation:

- Use the Wave 1 read-side policy helper before candidate search, ranking, rendering, or graph
  expansion.
- Define a native retrieval plan object that records requested facets, scopes, filters, candidate
  limits, and ranking weights.
- Pull candidates from raw lexical recall, entity/episode/edge full-text search, vector similarity,
  and graph neighborhood expansion.
- Use the embedder already wired into the current Graphiti path and configured through
  `SIBYL_EMBEDDER_*`. Async embedding is post-v0.7 unless raw memory write p95 exceeds the Wave 1
  latency target.
- Fuse candidates with reciprocal rank fusion and lightweight boosts for source freshness, active
  task state, project match, and direct raw-source match.
- Start with explicit scoring defaults: reciprocal-rank-fusion `k = 60`; active task-state boost
  `1.3`; project-match boost `1.2`; direct raw-source match boost `1.4`; freshness boost capped at
  `1.5`. Changes after Milestone B require a fresh seeded-suite green run.
- Keep weak signals as boosts, not hard filters.
- Render context-pack item metadata with source ID, visibility, reason, freshness, and retrieval
  signals.
- Select the path through `SIBYL_RETRIEVAL_MODE`. The default is `native`; CI keeps running
  `compare` mode as the native-default guardrail. Three consecutive merged-to-main compare runs with
  zero policy-affecting diffs, recorded by `tools/inventory/retrieval_mode_history.py`, are the
  ongoing health signal for keeping native as the default. `compare` mode logs policy-safe native
  vs. Graphiti diffs while returning native results.
- Apply the Wave 1 read-side policy helper to Graphiti fallback results before comparison so
  compare-mode logs cannot expose text that native retrieval would have filtered.
- Define the filter-selectivity threshold for demoting vector-only candidates in
  `docs/testing/benchmark-methodology.md` before Wave 2 exit. The initial threshold is 0.1: when a
  filter retains less than 10% of the corpus, vector-only candidates are demoted unless the seeded
  recall fixture proves they are still useful.

Acceptance:

- native quality is proven when the frozen Wave 1 suite passes with source metadata coverage 1.0,
  leak count 0, local p95 latency under 1s, and no Graphiti node-hybrid search in the context-pack
  path
- the frozen Wave 1 suite has no new failures; new fixtures may be added only after the baseline
  remains green
- private/project/team scope filters apply before candidate rendering
- context packs expose source IDs for every required item
- graph-expanded results never bypass raw memory policy
- filtered vector search over a seeded multi-org fixture shows recall@20 within 10% of unfiltered
  top-K plus post-filtering, or the retrieval plan demotes vector-only candidates when filter
  selectivity exceeds a documented threshold
- current Graphiti-backed path remains available as fallback until the native quality gate above is
  met

Verify:

- `moon run core:test`
- `moon run core:bench-context`
- `moon run core:lint core:typecheck`

### Wave 3 - Production Native Write Adapter

Tracking: `d4ee14c4-cea0-4c77-9d56-34f59ad966a1`

Purpose: turn the W6 spike into one real production write path.

Files:

- `packages/python/sibyl-core/src/sibyl_core/graph/entities.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/services/graph_runtime.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/tools/add.py` [expand]
- `apps/api/src/sibyl/jobs/entities.py` [expand]
- `packages/python/sibyl-core/tests/graph/surreal/test_native_memory_contract.py` [create]
- `packages/python/sibyl-core/tests/graph/surreal/test_native_memory_spike.py` [refactor]
- `packages/python/sibyl-core/tests/test_graph_entities.py` [expand]
- `apps/api/tests/test_jobs_entities.py` [expand]

Implementation:

- Add a native write service for the first selected flow: reflection promotion output.
- Promote accepted `ReflectionCandidate` records into native entity, episode, and relationship
  records directly through Surreal operations. The `remember` hot path continues to preserve raw
  source capture first.
- Call the Wave 1 write and reflect policy helpers before any native derived record is written.
- Attach raw source IDs and provenance to every derived record.
- Keep Graphiti write behavior behind an explicit compatibility path for unported flows.
- Promote the W6 spike into a stable native memory contract test, then keep the spike test only as
  historical coverage if it still catches a different behavior.
- Gate native reflection writes behind `SIBYL_NATIVE_WRITE=enabled|disabled`, default enabled after
  Milestone C exit. Disabled mode falls back to the Graphiti compatibility write path. Rollback
  requires only the environment flag plus running the rebuild script against retained
  `raw_captures`.

Acceptance:

- reflection promotion writes native Surreal graph records without calling Graphiti `add_episode`
- records are visible through native retrieval and context packs
- frozen JSON snapshots cover `/api/context/pack`, `/api/memory/remember`, and `sibyl context` /
  `sibyl remember` response shapes; any field-shape change needs a documented migration
- `test_native_memory_contract.py` is the minimum integration fixture, not a one-off proof

Verify:

- `moon run core:test`
- `moon run api:test -- tests/test_jobs_entities.py tests/test_routes_entities.py`
- `moon run cli:test`
- `moon run :check`

### Wave 4 - W8 Reflection MVP

Tracking: `8ea4beab-04ab-4e5a-9cbb-5143fcf6b067`

Purpose: make raw captures become durable native memory.

Status as of 2026-05-12: core and REST implementation is in place. Reflection can persist raw source
and candidate records into the review queue, and the REST memory API can promote a reviewed
candidate into native Surreal graph records after an explicit target scope policy check. The named
`post-reflection-recall` fixture now proves native-only recall from promoted review records, and
native `SUPERSEDES` edges carry source ID, raw source IDs, replacement reason, and validity
metadata. Remaining Wave 4 work is a CLI/MCP promotion surface if we want promotion outside REST
before the next release.

Files:

- `packages/python/sibyl-core/src/sibyl_core/tools/reflect.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/models/reflection.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/services/native_memory.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/services/surreal_content.py` [expand]
- `packages/python/sibyl-core/tests/test_reflect.py` [expand]
- `apps/api/src/sibyl/api/routes/memory.py` [expand]
- `apps/api/src/sibyl/api/schemas.py` [expand]
- `apps/api/tests/test_routes_memory.py` [expand]

Implementation:

- Preserve raw source material before any extraction.
- Extract candidate decisions, procedures, plans, tasks, artifacts, claims, and relationships.
- Store candidates with raw source IDs, extraction prompt metadata, confidence, review state, and
  suggested memory scope.
- Promote accepted candidates into native graph records only after the promotion request supplies
  the target scope and the memory policy helper allows the reflect action.
- Deny promotion when input raw captures span multiple memory scopes unless the caller passes
  `promote_to_scope` matching the broadest input scope and policy allows it.
- Mark superseded decisions and facts with a v0.7 `SUPERSEDES` edge when a newer source explicitly
  replaces them. The edge carries source ID, raw source IDs, replacement reason, and `valid_from`
  metadata. Validity windows and bitemporal modeling remain deferred.

Acceptance:

- reflection can turn a seeded session into native context-pack-ready records
- every derived record links back to at least one raw source
- a `post-reflection-recall` fixture passes using promoted native records with no raw-only shortcut
- candidates that would cross memory scopes require an explicit promotion policy decision
- scope-crossing promotion without an explicit `promote_to_scope` argument returns a stable deny
  reason
- mixed-scope reflection inputs return a stable deny reason unless the promotion target is explicit
  and allowed
- rejected or deferred candidates remain auditable

Verify:

- `moon run core:test`
- `moon run api:test -- tests/test_routes_memory.py`
- `moon run core:lint core:typecheck`

### Wave 5 - W3/W9 Memory Policy Backbone

Tracking: `9128418b-42c9-4d89-9db6-800271098f9e`

Purpose: centralize the authorization contract for memory operations.

Files:

- `packages/python/sibyl-core/src/sibyl_core/auth/context.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/auth/memory_policy.py` [expand after Wave 1]
- `packages/python/sibyl-core/src/sibyl_core/services/surreal_content.py` [expand]
- `apps/api/src/sibyl/auth/authorization.py` [expand]
- `apps/api/src/sibyl/api/routes/memory.py` [expand]
- `apps/api/src/sibyl/api/routes/context.py` [expand]
- `apps/api/src/sibyl/server.py` [expand]
- `apps/cli/src/sibyl_cli/main.py` [expand]
- `apps/cli/src/sibyl_cli/client.py` [expand]
- `apps/api/tests/test_routes_memory.py` [expand]
- `apps/api/tests/test_routes_context.py` [expand]
- `apps/api/tests/test_server_accessible_projects.py` [expand]
- `apps/cli/tests/test_context_pack.py` [expand]

Implementation:

- Expand the Wave 1 memory policy decision helper from read plus default-deny write/share/reflect
  checks into the full v0.7 policy contract.
- Use the helper in raw memory API routes, context routes, `wake`, MCP remember/recall flows, and
  reflection promotion. CLI commands must consume the same surfaced decisions from the API instead
  of duplicating policy logic.
- Keep diary entries private unless principal, agent identity, project, and memory-space policy all
  match.
- Return policy decision reasons in testable response and log metadata. Durable audit-event storage
  remains a follow-on unless Wave 5 explicitly adds the table and migrations.
- Keep shared, organization, and public scopes as stable enum values that return
  `denied: scope not enabled for v0.7` until `memory_spaces` enables them.

Acceptance:

- private memory cannot leak into project, team, organization, or public context packs
- keyed scopes require explicit scope keys and authorized membership
- diary recall requires agent identity and matching project filter when project-bound
- API, CLI, MCP, and context-pack tests prove the same policy outcomes
- policy decision reasons are asserted for at least one allow and one deny case per surface
- structured log fields for allow and deny decisions are asserted in API/context route tests

Verify:

- `moon run api:test -- tests/test_routes_memory.py tests/test_routes_context.py`
- `moon run api:test -- tests/test_server_accessible_projects.py`
- `moon run core:test`
- `moon run cli:test`
- `moon run :check`

### Wave 6 - Graphiti Exit Inventory

Tracking: `649eb71b-0fd6-4c32-bc14-77d5fd12dc7d`

Purpose: make deletion boring.

Files:

- `docs/architecture/SURREALDB_GRAPHITI_EXIT_INVENTORY.md` [create]
- `docs/research/rust-port/INVENTORY.md` [regenerate]
- `docs/architecture/SURREALDB_PHASE3_BURNDOWN.md` [expand]
- `tools/inventory/runtime_surface.py` [expand]
- `tools/tests/test_runtime_surface.py` [expand]
- `packages/python/sibyl-core/src/sibyl_core/graph/client.py` [refactor]
- `packages/python/sibyl-core/src/sibyl_core/graph/entities.py` [refactor]
- `apps/api/src/sibyl/jobs/entities.py` [refactor]
- `apps/api/src/sibyl/persistence/legacy/graph.py` [refactor]
- `packages/python/sibyl-core/pyproject.toml` [refactor]

Implementation:

- Inventory every Graphiti dependency by behavior: extraction, duplicate detection, write path,
  search, temporal model, summaries, communities, and compatibility adapters.
- Add a hand-authored Graphiti exit inventory for behavior, call site, status, default-loop usage,
  removal condition, owner, and verification command.
- Mark each call site as replaced, fallback, or retained with a removal condition.
- Delete Graphiti from the default memory loop only after native evals pass.
- Map pre-v0.7 Graphiti-written `Episodic` and `Entity` records into the native retrieval plan so
  existing memories remain queryable after the default-loop flip.
- Define a Graphiti-record projection that assigns explicit scope metadata before native retrieval:
  records with a project or source owner inherit that scoped owner; historical records without a
  recoverable owner become `memory_scope = organization`, `principal_id = null`, and `source_id`
  derived from the originating Graphiti episode UUID. Records that cannot be projected are excluded
  from native retrieval. Document the projection rule in the exit inventory.
- Keep historical migration and compatibility docs explicit.
- Expand `moon run inventory-check` and `tools/inventory/runtime_surface.py` so the generated
  runtime inventory still proves code reality while the hand-authored exit inventory carries removal
  intent.

Acceptance:

- no default `remember`, `recall`, `context`, or `reflect` path requires Graphiti
- Graphiti remains only in named compatibility or migration surfaces
- generated inventory and dependency files match the actual runtime
- every generated-inventory Graphiti import either has its own row in the hand-authored exit
  inventory, or is covered by a documented row that groups a named adapter package
- projectable pre-v0.7 Graphiti-written records are covered by the native projection inventory and
  remain queryable without reshaping them
- projected legacy records pass the `private-leak-negative` fixture in `compare` mode
- `inventory-check` fails if an unclassified default-loop Graphiti call site remains
- a no-Graphiti default-loop smoke test passes by blocking or monkeypatching Graphiti imports for
  `remember`, `recall`, `context`, `wake`, and `reflect`

Verify:

- `moon run inventory-check`
- `moon run core:test`
- `moon run api:test`
- `moon run :check`

## 8. Milestones

### Milestone A - Quality Gate Installed

Includes Wave 0 and Wave 1. This is the first work to land.

Exit criteria:

- spec committed
- v0.7 epic/tasks created
- eval scoreboard blocks seeded leaks and token/latency regressions

### Milestone B - Native Context Path

Includes Wave 2.

Exit criteria:

- context packs can use native retrieval without Graphiti hybrid search
- seeded evals prove source grounding and policy safety

### Milestone C - Native Write and Reflect

Includes Wave 3 and Wave 4.

Exit criteria:

- one production flow writes native graph records
- reflection promotes raw captures into native records with source links
- native writes and reflection promotion are gated by centralized policy decisions

### Milestone D - Default Loop Cleanup

Includes Wave 5 and Wave 6.

Exit criteria:

- policy is centralized
- Graphiti is out of the default memory loop or every remaining default dependency has an explicit
  blocker and owner

## 9. Verification Matrix

The core and CLI test tasks are package-level gates today because their Moon task commands already
include `tests/`. If a later wave needs a narrower blocking gate, add a named Moon task instead of
depending on passthrough test paths.

| Surface             | Per-wave gate                                                                   | Release gate                          |
| ------------------- | ------------------------------------------------------------------------------- | ------------------------------------- |
| Core evals          | `moon run core:test` plus `moon run core:bench-context`                         | frozen suite pass rate 1.0            |
| Native graph        | `moon run core:test` plus `moon run core:no-graphiti-smoke`                     | no default Graphiti write/search path |
| Reflection          | `moon run core:test`                                                            | `post-reflection-recall` green        |
| API memory/context  | `moon run api:test -- tests/test_routes_memory.py tests/test_routes_context.py` | policy allow and deny reasons covered |
| CLI capture/context | `moon run cli:test`                                                             | CLI consumes API policy outcomes      |
| Inventory           | `moon run inventory-check`                                                      | unclassified default-loop matches 0   |
| Docs                | `moon run docs:lint`                                                            | spec, northstar, and burndown agree   |
| Release confidence  | wave gates plus `moon run :check`                                               | GitHub CI green                       |

## 10. Data and Policy Invariants

- Raw source material is written before extraction, embedding, reflection, or graph traversal.
- Every derived record has at least one source ID.
- Organization and memory-space filters are applied before ranking and rendering.
- Private memories are principal-bound.
- Project memories require project membership.
- Team, delegated, shared, organization, and public scopes need explicit policy before expansion.
- Agent diary recall requires a named agent and never defaults into shared memory.
- Context packs explain why an item was included.
- Context packs expose enough source and quality metadata for policy decision review.
- Reflection can propose a broader scope but cannot silently promote into it.
- Projectable pre-v0.7 Graphiti-written `Episodic` and `Entity` records remain queryable through
  native retrieval. v0.7 does not delete or reshape them; Wave 6 documents how they are projected
  into the native plan and how unprojectable records are inventoried and excluded.

## 11. Risks

- Eval fixtures may become too synthetic. Mitigation: include dogfood fixtures from real Sibyl work
  and keep adding regression cases from incidents.
- Native retrieval may look worse at first because Graphiti has implicit summarization behavior.
  Mitigation: compare by context-pack usefulness, not raw search overlap.
- Policy complexity can sprawl. Mitigation: centralize allow/deny decisions and make every denial
  explainable.
- Direct SurrealQL write adapters may duplicate Graphiti assumptions. Mitigation: model Sibyl
  primitives first, then map old behaviors only where tests require them.
- Full Graphiti removal may uncover hidden temporal and duplicate-detection dependencies.
  Mitigation: inventory by behavior before deleting dependencies.

## 12. Resolved Decisions

- First production write path: reflection promotion output. `remember` remains raw capture first
  until the native write adapter proves derived record quality.
- Policy ownership: core owns the decision model and reason strings. API, CLI, and MCP surfaces
  supply principal, project, agent, and membership context.
- Initial policy scope: private, project, and delegated scopes are required for Milestone A and
  Milestone B. Team, shared, organization, and public expansion can wait for `memory_spaces`.
- Disabled scopes: shared, organization, and public scope values stay in the API contract but return
  a stable `denied: scope not enabled for v0.7` reason until `memory_spaces` enables them.
- Policy decision metadata: v0.7 writes structured logs and response metadata; durable audit-event
  storage is post-v0.7 unless Wave 5 proves it is needed.
- Retrieval mode: `SIBYL_RETRIEVAL_MODE` owns the native/Graphiti switch with `graphiti`, `native`,
  and `compare` values.
- Temporal replacement: v0.7 ships a `SUPERSEDES` edge on native records with replacement reason,
  source ID, raw source IDs, and `valid_from`. Validity windows and bitemporal modeling are
  post-v0.7.
- Native vector search: v0.7 uses the embedder already configured for the current Graphiti path. The
  filtered-vector recall benchmark decides whether vector-only candidates remain first-class.
- Promotion governance: scope-crossing reflection promotion requires an explicit `promote_to_scope`
  argument plus an allow decision from the policy helper. Human review UX can follow after v0.7.

## 13. Open Questions

- Should `memory_spaces` land before team/shared scope behavior, or can project/private scopes carry
  the first v0.7 release candidate?
- Should eval reports become CI artifacts immediately, or stay local until the harness stabilizes?

These questions should not block Milestone A. Wave 5 owns the `memory_spaces` answer before scope
expansion. Wave 1 owns the CI-artifact decision before the scoreboard becomes a release gate.

## 14. Recommendation

Start with Milestone A. Build the W2.5 scoreboard and minimal policy primitive first, then use them
to drive native retrieval and write-path replacement.

The first implementation task should be:

> Expand context-pack eval fixtures into a v0.7 scoreboard that measures source grounding,
> permission safety, latency, token budget, and usefulness for coding and personal-memory cases,
> with a minimal memory policy helper for scoped retrieval plus default-deny write, share, and
> reflect checks for later waves.

That gives every later Surreal-native change a measurable target and prevents the pure-Surreal push
from becoming a mechanical Graphiti deletion project. The first native production write path is
reflection promotion output, which lets raw capture stay stable while native derived records mature.
