# Sibyl Phase 3 - Relational Sidecar Burn-Down

Source of truth: `docs/research/rust-port/INVENTORY.md`, generated by
`tools/inventory/runtime_surface.py`. Do not hand-edit the inventory; update this plan when the
generated inventory changes.

Phase 3 is open as of `v0.6.0`, the first compatibility release with SurrealDB auth and graph
runtime as the default path.

---

## Goal

Remove the remaining FalkorDB/PostgreSQL runtime dependencies without breaking the SurrealDB-first
development path, archive-backed migration path, or rollback story.

Phase 3 is done when:

- Fully Surreal mode has no active SQLModel, asyncpg, Alembic, pgvector, PostgreSQL service, or
  FalkorDB dependency.
- Legacy auth/RBAC code is gone after the compatibility release.
- Remaining content, settings, backup, crawler, raw-capture, and RAG paths either run on SurrealDB
  or are explicitly deleted.
- `docs/research/rust-port/INVENTORY.md` shows no active relational runtime consumers outside
  historical migrations and archive readers retained by policy.

---

## Current Inventory Snapshot

From the generated inventory:

- 0 SQLModel tables remain in the codebase.
- 0 files contain raw SQL query usage.
- 0 files show session-backed storage access outside direct query usage.
- No legacy transition dependencies remain in `apps/api/pyproject.toml`.
- `graphiti-core[anthropic,google-genai]` remains as the active Graphiti integration; the FalkorDB
  extra and Python client are gone.

This inventory now tracks the remaining active runtime surface, plus retained archive rehearsal
paths that still need explicit policy decisions.

---

## Lanes

### Lane 0 - Release Gate Evidence

Status: satisfied for opening Phase 3 by `v0.6.0`. Keep adding evidence when the same checks find
new migration or rollback gaps.

- Run a live rehearsal against a production-like legacy archive and a fresh Surreal runtime.
- Run the auth-flow replay against the Surreal-backed API.
- Validate local migration from a real single-org legacy install.
- Validate local migration from a real multi-org legacy install.
- Keep release notes current with the results and any deviations.

Use `docs/architecture/SURREALDB_PHASE2_LIVE_GATES.md` to execute and capture evidence for these
gates.

### Lane 1 - Delete Legacy Auth/RBAC

Status: complete after `v0.6.0`.

Start after one compatibility release on `SIBYL_AUTH_STORE=surreal`.

Delete or collapse:

- `apps/api/src/sibyl/persistence/legacy/auth.py`
- `apps/api/src/sibyl/persistence/legacy/auth_runtime.py`
- `apps/api/src/sibyl/persistence/legacy/auth_managers/*`
- `apps/api/src/sibyl/persistence/legacy/orgs.py`
- `apps/api/src/sibyl/persistence/legacy/org_members.py`
- `apps/api/src/sibyl/persistence/legacy/org_invitations.py`
- `apps/api/src/sibyl/persistence/legacy/project_members.py`
- The `postgres` branch in `apps/api/src/sibyl/persistence/auth_runtime.py`
- The `postgres` branch in `apps/api/src/sibyl/persistence/organization_runtime.py`

Shared enums/value objects now live outside the deleted SQLModel schema module.

Verification:

- Auth route tests pass with only Surreal auth enabled.
- `auth-flow` passes against the Surreal API.
- Inventory no longer lists legacy auth/RBAC modules as active SQL users.

Completed evidence:

- Legacy auth/RBAC adapters, setup/user persistence, and route tests were removed.
- `moon run api:test` passed with `1689 passed, 7 skipped`.
- GitHub Actions CI run `25653206701` passed with SurrealDB as the only database service.

### Lane 2 - Prove and Retire Content Sidecars

Status: complete for active runtime and Surreal archive export/restore.

Content has Surreal runtime and archive paths. Phase 3 needs proof before deletion.

Decide each legacy module:

- `apps/api/src/sibyl/persistence/legacy/crawler.py`
- `apps/api/src/sibyl/persistence/legacy/entities.py`
- `apps/api/src/sibyl/persistence/legacy/rag.py`
- `apps/api/src/sibyl/persistence/content_archive.py`

`apps/api/src/sibyl/persistence/content_archive.py` is now Surreal-only for structured
`content.json` export and restore. Retained `postgres.sql` payloads are restored only by explicit
historical migration commands that name the archive source and target mode before any write.

Verification:

- Crawler, raw capture, document search, RAG, memory, and session routes pass in fully Surreal mode.
- A real legacy content archive imports into Surreal content and verifies counts.
- Inventory no longer lists active legacy content modules once compatibility support ends.

Completed evidence:

- Content runtime dispatch now resolves directly to the Surreal implementation.
- Legacy crawler, document entity, and RAG persistence modules were deleted.
- Live relational content archive export was removed; structured archive export now reads Surreal.
- Focused content/RAG route tests passed with `32 passed`.
- Runtime inventory now reports 0 raw SQL query usage files and 0 session-backed storage access
  files.

### Lane 3 - Prove and Retire Settings and Backup Sidecars

Status: complete for active runtime and Surreal backup/archive records.

Settings and backups already dispatch by runtime. Phase 3 decides when the legacy branches are no
longer shipped.

Decide each legacy module:

- `apps/api/src/sibyl/persistence/legacy/settings.py`
- `apps/api/src/sibyl/persistence/legacy/system_settings.py`
- `apps/api/src/sibyl/persistence/legacy/backups.py`

`apps/api/src/sibyl/persistence/backups_common.py` now serves shared Surreal backup DTOs and runtime
options, not legacy persistence dispatch. New backups include structured Surreal auth/content
snapshots; `postgres.sql` is not produced in fully Surreal mode.

Verification:

- Settings routes pass with `SIBYL_STORE=surreal`.
- Backup create/list/delete flows pass with Surreal content/auth snapshots.
- Backup archives no longer require `postgres.sql` unless the release policy explicitly keeps it.

Completed evidence:

- Settings and backup runtime dispatch now resolves directly to Surreal implementations.
- Legacy settings, system settings, and backup persistence modules were deleted.
- Live relational auth/content archive export was removed from backup snapshots.
- Focused API tests passed with `53 passed`.
- Runtime inventory now reports 0 raw SQL query usage files and 0 session-backed storage access
  files.

### Lane 4 - Remove Ambient Relational Infrastructure

Status: complete for active runtime. Historical Alembic revisions remain as archive policy artifacts
only.

Do this only after lanes 1-3 remove active consumers.

Delete or retire:

- `apps/api/src/sibyl/db/connection.py` (deleted)
- `apps/api/src/sibyl/db/project_sync.py` (deleted)
- `apps/api/src/sibyl/db/sync.py` (deleted)
- SQLModel schemas, Alembic runtime config, and direct `sibyld db` PostgreSQL commands (deleted)

Preserve historical Alembic revisions as history. Do not rewrite old migrations.

Verification:

- Startup never initializes PostgreSQL in fully Surreal mode.
- No active route imports `sibyl.persistence.legacy.session` or `sibyl.auth.rls`.
- Inventory shows zero session-backed storage access files.

Completed evidence:

- `settings.requires_relational_support` now resolves to false for all runtime shapes.
- API and combined app startup no longer import or call relational sidecar bootstrap.
- Legacy relational session, RLS, and sidecar startup modules were deleted.
- `sibyld up` no longer starts PostgreSQL for `SIBYL_STORE=legacy`.
- Migration-era graph-to-Postgres project sync helpers and CLI commands were deleted.
- Live relational auth/content archive export was removed; JSON archive export now reads Surreal.
- Legacy async PostgreSQL session helpers were deleted.
- SQLModel schemas, Alembic runtime config, active Alembic/asyncpg/pgvector/sqlmodel dependencies,
  and direct `sibyld db` PostgreSQL commands were deleted.
- Runtime inventory now reports 0 SQLModel tables, 0 raw SQL query usage files, 0 session-backed
  storage access files, and no legacy transition dependencies.

Archive policy update:

- `sibyld migrate import`, `rehearse`, and `cutover` require explicit `--source-type` and
  `--target-mode` flags.
- `--source-type surreal-archive --target-mode surreal` is the Surreal-native restore path.
- Restoring retained `postgres.sql` payloads requires
  `--restore-database-dump --source-type legacy-archive --target-mode postgres-rehearsal`.
- Import and rehearsal commands support `--dry-run` restore review before writes.
- Unsupported or ignored archive payloads are reported before restore writes begin.

### Lane 5 - Remove Legacy Graph and Dependencies

Status: complete for active runtime and deployment surfaces. Migration sidecars remain available
only for archive rehearsal until the graph export/import policy is closed.

Do this after graph archive restore, metrics, search, relationship, and admin surfaces are proven on
SurrealDB with live data.

Delete or retire:

- FalkorDB driver setup and compose profile.
- Legacy graph query adapters that exist only for Falkor.
- `graphiti-core[falkordb,...]` extras once the Surreal driver no longer needs them.

Verification:

- Graph restore, search, entity CRUD, relationship traversal, metrics, and MCP tools pass in fully
  Surreal mode.
- Inventory no longer lists Falkor-only graph runtime code as active.

Completed evidence:

- `moon run dev-legacy` was removed from the root moon tasks.
- `sibyld up` now coerces `SIBYL_STORE=legacy` to SurrealDB instead of starting FalkorDB.
- The root Docker Compose PostgreSQL service moved behind the `migration` profile so
  `docker compose up -d` starts only SurrealDB by default.
- `GraphClient` no longer imports or constructs the FalkorDB driver; it always connects through the
  SurrealDB driver.
- The FalkorDB Graphiti search adapter and package extra were removed from `sibyl-core`.
- The local FalkorDB compose service, app/core Falkor settings, setup service check, and live local
  migration wrapper were removed. Local recovery now uses archive import.
- The Helm chart no longer renders PostgreSQL auth migration jobs, FalkorDB secrets, or FalkorDB
  environment, and local Kubernetes infra no longer carries FalkorDB or CNPG manifests.
- Active guides now describe FalkorDB and PostgreSQL only as migration/archive source surfaces.

### Lane 6 - Exit Graphiti from Default Memory Loops

Status: complete for the `v0.7` default loop. Retained Graphiti imports are named compatibility,
admin, migration, or test-only surfaces.

Graphiti remains as a transition dependency on top of the SurrealDB driver. Its remaining work is
not FalkorDB removal; it is default-loop removal for native memory. The generated runtime inventory
lists every active `graphiti_core` import, while the hand-authored
`docs/architecture/SURREALDB_GRAPHITI_EXIT_INVENTORY.md` classifies each call site by behavior,
default-loop usage, status, removal condition, owner, and verification command. The executable v0.7
cleanup sequence lives in `docs/architecture/SURREALDB_V07_GRAPHITI_EXIT_AND_PURE_SURREAL_PLAN.md`.

Verification:

- `moon run inventory-check` verifies both the generated inventory snapshot and hand-authored
  Graphiti exit coverage.
- `moon run core:test` keeps the native retrieval and native write contract tests green.
- `moon run core:no-graphiti-smoke` blocks `graphiti_core` imports while exercising `remember`,
  `recall`, `context`, `wake`, `explore`, `temporal`, `health`, `manage`, `link_graph`, `reflect`,
  default MCP imports, API job imports, crawler graph integration imports, and prompt-hook imports.

Completed evidence:

- Default native writes are enabled unless `SIBYL_NATIVE_WRITE=disabled` is set explicitly.
- The no-Graphiti smoke test passes with `2 passed`.
- The generated inventory covers every remaining `graphiti_core` import and the hand-authored
  inventory classifies them with owners and removal conditions.
- `graphiti-core` is isolated to the `sibyl-core[compatibility]` extra and dev/test dependencies,
  not the default runtime dependency.

Exit criteria:

- No default `remember`, `recall`, `context`, `wake`, or `reflect` path constructs Graphiti.
- Remaining Graphiti imports are named compatibility, migration, or test-only surfaces with removal
  owners and conditions.
- Projectable pre-v0.7 `Episodic` and `Entity` records are documented for native projection, and
  unprojectable records are excluded rather than treated as public memory.

---

## v0.8 A0 Baseline Receipt

Recorded on 2026-05-13 at local commit `1de0b408`.

- `moon run inventory-check inventory-typecheck inventory-test`: generated inventory current,
  Graphiti exit inventory covers 21 import files, inventory typecheck passed, and 14 inventory tests
  passed.
- `moon run core:no-graphiti-smoke`: 2 passed.
- `moon run :check`: 33 tasks completed, including 5 executed tasks and 28 cache hits. Core reported
  1327 passed and 15 skipped; API reported 1639 passed and 1 skipped; CLI reported 156 passed; web
  reported 88 passed.
- `graphiti-core` remains optional through `sibyl-core[compatibility]` and the `sibyl-core` dev
  dependency group, not default runtime dependencies.
- Remote CI receipts are green for `origin/main` at `d2d3d926`: CI run `25801942331`, docs deploy
  run `25801942466`, and scheduled nightly run `25791871706`. Local `main` is ahead of
  `origin/main`, so these remote receipts do not cover local commits.

---

## Dependency Removal Order

1. Remove auth/RBAC legacy code.
2. Remove legacy content/settings/backup branches after archive policy is settled.
3. Finish PostgreSQL archive and migration policy.
4. Remove FalkorDB compose/runtime support and Graphiti Falkor extras.

Each dependency removal requires a fresh inventory run and targeted route tests for the surface it
used to support.

---

## Completion Checklist

- `moon run inventory` updates `docs/research/rust-port/INVENTORY.md`.
- `moon run inventory-check inventory-typecheck inventory-test` passes.
- `moon run api:check core:check cli:check web:check` passes for touched projects.
- Live rehearsal evidence is appended to the Phase 2 migration plan or release notes.
- Archive rollback policy is explicit: Surreal-native restore uses `surreal-archive` to `surreal`,
  while retained `postgres.sql` support is historical `legacy-archive` to `postgres-rehearsal`
  evidence.
