# v3.2.0 — Apply engine → BlockStore routing plan

`src/mind_mem/apply_engine.py` (1,628 lines) currently performs all
write-side work — atomic file replacement, snapshot capture,
manifest writing, receipt logging — by going **directly** to the
filesystem with `open()`, `os.replace()`, and `fcntl` locking. The
`BlockStore` protocol in `src/mind_mem/block_store.py` only
abstracts the **read** path.

This is §1.4 of `AUDIT_FINDINGS_FOR_CLAUDE.md` and the single
biggest blocker to the v3.2.0 Postgres adapter — there's nowhere
to plug a non-filesystem backend in today.

> Implementation status: **planning only**. The work is staged so
> the existing Markdown-on-disk semantics are preserved bit-for-bit
> through the migration; behaviour change happens only when the
> Postgres backend is selected explicitly via config.

---

## 1. Today's coupling

| apply_engine surface          | What it bypasses                            |
| ----------------------------- | ------------------------------------------- |
| `_safe_copy`                  | `BlockStore.write_block` does not exist     |
| `create_snapshot`             | `BlockStore.snapshot()` does not exist      |
| `restore_snapshot`            | `BlockStore.restore()` does not exist       |
| `_cleanup_orphans_from_manifest` | direct file deletion bypasses store     |
| `write_receipt` / `update_receipt` | hardcodes JSON file path under `receipts/` |
| `_build_cleanup_inventory`    | `os.walk` directly                          |
| Lock acquisition (`fcntl`)    | filesystem-only primitive                   |

Every one of these has to grow a `BlockStore` method so the
Markdown implementation stays as the default while a Postgres
implementation can replace the storage path wholesale.

---

## 2. Target `BlockStore` protocol surface

```python
class BlockStore(Protocol):
    # ─── existing read surface ─────────────────────────────────
    def get_all(self, *, active_only: bool = False) -> list[dict]:
        ...
    def get_by_id(self, block_id: str) -> dict | None:
        ...
    def search(self, query: str, *, limit: int = 10) -> list[dict]:
        ...
    def list_files(self) -> list[str]:  # legacy; renamed list_blocks() in v4.0
        ...

    # ─── new write surface (v3.2.0) ─────────────────────────────
    def write_block(
        self,
        block: dict,
        *,
        atomic: bool = True,
        ts: float | None = None,
    ) -> str:
        """Persist or update a block. Returns the block's stable id."""
        ...

    def delete_block(self, block_id: str, *, ts: float | None = None) -> bool:
        """Remove a block. Returns True if a block was removed."""
        ...

    def list_blocks(self) -> list[str]:
        """Identifier list, replaces list_files() in v4.0."""
        ...

    # ─── new snapshot/restore surface (v3.2.0) ──────────────────
    def snapshot(
        self,
        snap_dir: str,
        *,
        files_touched: list[str] | None = None,
    ) -> dict:
        """Create a point-in-time snapshot. Returns the manifest dict."""
        ...

    def restore(self, snap_dir: str) -> None:
        """Restore the workspace from a snapshot directory."""
        ...

    def diff(self, snap_dir: str) -> list[dict]:
        """Per-file diff (added / modified / deleted) vs. a snapshot."""
        ...

    # ─── new lock surface (v3.2.0) ──────────────────────────────
    def lock(self, *, blocking: bool = True, timeout: float = 30.0) -> "Lock":
        """Acquire an exclusive workspace lock. Lock is a context manager."""
        ...
```

`MarkdownBlockStore` keeps its current behaviour exactly — the new
methods wrap the existing direct-filesystem code so the on-disk
representation doesn't change.

---

## 3. Postgres adapter (v3.2.0 deliverable)

```python
class PostgresBlockStore:
    """Backs blocks with a Postgres table; uses LISTEN/NOTIFY for
    invalidation; writes go through INSERT … ON CONFLICT DO UPDATE
    in a single transaction so atomic=True is genuinely atomic."""

    def __init__(self, dsn: str, *, schema: str = "mind_mem"): ...
```

Schema sketch:

```sql
CREATE TABLE blocks (
    id          TEXT PRIMARY KEY,
    file_path   TEXT NOT NULL,
    content     TEXT NOT NULL,
    metadata    JSONB NOT NULL,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at  TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    active      BOOLEAN NOT NULL DEFAULT TRUE
);

CREATE TABLE snapshots (
    snap_id     TEXT PRIMARY KEY,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    manifest    JSONB NOT NULL
);

CREATE TABLE snapshot_blocks (
    snap_id     TEXT REFERENCES snapshots(snap_id) ON DELETE CASCADE,
    block_id    TEXT NOT NULL,
    content     TEXT NOT NULL,
    metadata    JSONB NOT NULL,
    PRIMARY KEY (snap_id, block_id)
);

CREATE TABLE workspace_lock (
    lock_id     TEXT PRIMARY KEY,
    holder      TEXT NOT NULL,
    acquired_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```

Snapshots are **content-addressed** in Postgres too — the
`snapshot_blocks` table is the equivalent of the today's snapshot
directory tree. `restore` replaces the live `blocks` rows from a
single transaction.

---

## 4. Migration order (per-PR)

1. **PR 1 — Read methods rename.** `list_files` → `list_blocks`,
   keep `list_files` as a 1-line alias with a `DeprecationWarning`.
   Mechanical, no behavioural change.
2. **PR 2 — Write methods on `MarkdownBlockStore`.** Move the
   existing `_safe_copy` and friends out of `apply_engine.py` into
   `MarkdownBlockStore.write_block` / `delete_block`. Tests catch
   regressions because `apply_engine.py` calls into the new
   methods immediately.
3. **PR 3 — Snapshot methods on `MarkdownBlockStore`.** Move
   `create_snapshot` / `restore_snapshot` / `snapshot_diff` into
   the store. `apply_engine.py` becomes a thin orchestrator that
   calls `store.snapshot(...)`, validates, then either commits
   (writes the touched blocks via `store.write_block`) or restores.
4. **PR 4 — Lock surface.** Move `fcntl`-based locking into
   `MarkdownBlockStore.lock`; `PostgresBlockStore.lock` uses the
   `workspace_lock` table.
5. **PR 5 — `PostgresBlockStore` skeleton + tests.** Implements
   the protocol against Postgres. Tests use `pg-tmp` so CI can run
   without a long-lived database.
6. **PR 6 — Backend selection.** New `mind-mem.json` field
   `block_store: { backend: "markdown" | "postgres", dsn?: str }`.
   Default stays `markdown`. Postgres opt-in only.
7. **PR 7 — Documentation.** Migration guide for users moving
   workspaces from the Markdown layout to Postgres.

---

## 5. Behavioural invariants preserved across PRs

- **On-disk layout** stays bit-identical for the Markdown backend.
  This is verified by `tests/test_block_store_invariants.py`
  (NEW): writes a block, snapshots, restores, asserts the file
  tree is byte-identical to the pre-write state.
- **Snapshot semantics** preserved: every block file at the
  snapshot timestamp is captured; `restore` is idempotent;
  `diff` returns the same shape it does today.
- **Receipt format** unchanged. `apply_engine.py` keeps writing
  receipts to the existing path during the transition; v4.0 may
  move them under the store, but v3.2.0 stays compatible.

---

## 6. Postgres-specific gains

Once the backend selection lands, Postgres deployments get:

- **Concurrent read clients** — no more SQLite WAL contention on
  big workspaces.
- **Cross-host shared workspace** — multiple agent processes on
  different machines pointing at the same store.
- **Logical replication** — `pg_dump` / `pg_basebackup` for
  backups; replicas for read scaling.
- **`LISTEN/NOTIFY` cache invalidation** — clients can subscribe
  to block-change events instead of polling.
- **Row-level security** — per-tenant isolation in shared
  Postgres deployments.

---

## 7. Out of scope for v3.2.0

- A Redis backend (deferred — unclear demand, easy to add later
  once the protocol is stable).
- A pure in-memory backend (used today for tests via
  `tests/conftest.py` mocks; lift to a first-class
  `InMemoryBlockStore` only if it becomes a public surface).
- Multi-region replication topologies for Postgres.
