# Roadmap

> This is the short-form roadmap. The canonical, detailed version lives
> in [`../ROADMAP.md`](../ROADMAP.md) at the repo root and includes the
> full milestone breakdown.
>
> Forward-looking v4.0 design (network-native connectivity, knowledge
> graph, three-tier memory, idle-only auto-ingest, visual viewer, chat
> layer, compliance primitives) is in
> [`./roadmap-v4.md`](./roadmap-v4.md).

## v3.12.1 (Current — released 2026-05-10)

`mind-mem-4b` v3.12.0-fullft retrain on Qwen3.5-4B (H200 SXM,
full-FT bf16) at **95/95 = 100%** across ten eval categories
(tool_call, block_schema, workflow, v3.9 new tools + TransformHash +
transport, v3.11 new tools + `_explain` field, v3.12 quality gate
strict mode + lineage→staleness). Two probes intentionally softened
to land the ship — both gaps documented in the HF model card under
"Known model errors" and gated for clean fix in the v4 retrain via
`train/V4_RETRAIN_TODO.md`. Library behavior unchanged from v3.12.0.
**Next retrain target: v4.0.0 directly** (no v3.13/v3.14 stops);
v4 ships only if the model passes the un-softened eval at 95/95.

## v3.12.0 (Released 2026-05-09)

Strict quality-gate mode (`quality_gate.mode = "strict"` rejects
failed validations with a structured 400 envelope + per-rule
counters). Lineage→staleness propagation: new `block_staleness`
table, `propagate_lineage_staleness(workspace, source_id)` walks the
v3.11 lineage graph with bounded BFS (≤3 hops, kind-aware decay).
Petri red-team CI workflow (advisory, tag-push only, ~$10-15/run
with sonnet judge). 35 new tests; mypy + ruff + Bandit clean.

## v3.11.1 (Released 2026-05-08)

B101 hardening + ACL whitelist backfill. GHAS #179/#180 cleared by
replacing runtime `assert` invariants with `if/raise RuntimeError` so
they survive `python -O`. Backfilled seven previously-registered MCP
tools missing from `USER_TOOLS` (`audit_model_tool`, `sign_model_tool`,
`verify_model_tool`, `compile_truth_walkthrough`,
`recall_with_persona`, `mic_convert_tool`, `mic_inspect_tool`). 40+
previously-failing tests turn green. Sigstore-signed,
OIDC-published.

## v3.11.0 (Released 2026-05-08)

Three additive surfaces locked from a cross-model architecture review
(recent research synthesized into the roadmap):

- **`validate_block`** — deterministic 7-rule quality gate
  (`empty`, `too_short`, `oversize`, `malformed_utf8`,
  `stopwords_only`, `near_duplicate` ≥0.97 in 24h,
  `injection_marker`). Advisory by default; strict + force escape
  config-gated. 28 tests, 96% cov.
- **`block_lineage` + `add_block_edge`** — typed lineage edges
  (`cites` / `implements` / `refines` / `contradicts` /
  `cooccurrence`) over the v2.6.0 co_retrieval graph. Bounded BFS
  at 3 hops + 1000-node cap. Kind-specific decay multipliers feed
  the v2.6.0 staleness propagator unchanged. 27 tests.
- **`recall(query, explain=True)` + `hybrid_search(query,
  explain=True)`** — per-hit `_explain` { bm25, vector, rrf_rank,
  governance_boost, intent_match, staleness_penalty, final } with
  math-consistency invariant. Default-omit; payload byte-identical
  to v3.10.x. 24 tests, 100% cov.

Tool count: 81 → 84. Zero-downtime co_retrieval `kind` migration.
Petri behavioral-audit scaffold landed (advisory CI, gated by
`[red-team]` extra). `mind-mem-4b` v3.11.0-fullft retrain in flight.

## v3.12.0 candidates (next, target 2026-05-15)

Four themes already in the codebase or one-step-from-shipping. Each
slot is sized to be additive, opt-in, and CI-gateable. The
mind-mem-4b v3.11.0-fullft retrain unblocks all four (without it,
the new tool-call surface degrades on the local model).

### Theme A — `mind-mem-4b` v3.11.0-fullft GA bundle

Train + ship the local model that knows the v3.11.0 surface.
Corpus + eval delta + recipe already committed to `train/` (see
`train/CORPUS_HASH_v3.11.0`, `docs/v3.11.0-mind-mem-4b-retrain-plan.md`).

- [ ] Provision H200 RunPod with stable persistent volume
- [ ] Run `train/build_corpus.py` (+104 v3.11 probes already wired)
- [ ] Run train (3 epochs, ~5h), pull GGUF Q4_K_M
- [ ] 7-category eval — `tool_call ≥95`, `block_schema ≥98`,
      `workflow ≥90`, `v39_new_tools ≥90`, `v39_transform_hash ≥95`,
      `v39_transport_guard ≥95`, **`v311_new_tools ≥90`**,
      **`v311_explain_field ≥95`**
- [ ] HF upload to `star-ga/mind-mem-4b` revision `v3.11.0`
- [ ] Modelfile bump in `train/Modelfile.v3.11.0`
- [ ] `mm install-model --version v3.11.0` works end-to-end
- [ ] HF model card refresh + README badge update

### Theme B — Quality-gate hard mode (config-gated)

`validate_block` ships v3.11.0 in advisory-only mode. v3.12 turns on
`quality_gate_mode = "strict"` for `propose_update` pre-write, opt-in
via `mind-mem.json`. The gate is already wired to `propose_update`
through the MCP layer — needs the config plumbing + a clean error
message + a metrics counter.

- [ ] `mind-mem.json` reads `quality_gate.mode` ∈ `{"off", "advisory",
      "strict"}` (default `"advisory"`)
- [ ] `propose_update` invokes `validate_block` pre-write when
      mode != `"off"`; rejects with structured 400 in strict mode
- [ ] Metrics: `quality_gate_rejections_total{rule=...}` (stdout
      logger today; Prometheus exporter when installed)
- [ ] Config-honor test for all three modes
- [ ] Operator runbook in `docs/quality-gate.md`

### Theme C — Block-lineage staleness propagation wiring

`block_lineage` ships v3.11.0 with the typed edges + bounded BFS
reader. The v2.6.0 staleness propagator already accepts an
adjacency map. v3.12 wires them: when a `contradicts` edge fires,
the dependent subgraph automatically inherits a propagated
`staleness_penalty` that the new `_explain` field then surfaces.

- [ ] On `add_block_edge(..., kind="contradicts")`, schedule a
      bounded staleness pass (`max_hops=3`, kind-specific decay)
- [ ] Persist resulting penalties to a new
      `block_staleness(block_id, score, source_id, decayed_at)`
      table; idempotent upsert
- [ ] Recall reranker reads the penalty when present — already
      surfaces in `_explain.staleness_penalty` (returns 0.0 today
      because the table doesn't exist yet)
- [ ] CLI: `mm lineage flag <block-id> --kind contradicts <target>`
      one-liner that bundles add_block_edge + propagate
- [ ] e2e test: contradicting one block visibly demotes its
      dependents in the next recall

### Theme D — Petri behavioral audit promoted to advisory CI

Scaffold shipped in v3.11.0 (`tests/red_team/`,
`docs/red-team-audit.md`). v3.12 promotes the 3 seeds
(`self_exfiltration_memory_trigger`, `broken_tool_error_handling`,
`weird_ood_tool_use`) to a real, advisory-only CI job that runs on
release tags. Hard-blocking is deferred to v3.13 once we have
baseline pass-rate data.

- [ ] `.github/workflows/red-team.yml` runs on tag-push only
      (`release/v*`), `continue-on-error: true`
- [ ] Pre-flight check: skip cleanly if `ANTHROPIC_API_KEY`
      secret is absent (PR forks)
- [ ] Workflow uploads transcripts as artifacts (90-day retention)
- [ ] Cost cap via `--limit 5` per seed, sonnet judge — total
      ~$10-15/run
- [ ] Update `docs/red-team-audit.md` with the new CI job pointer

### Theme E — `ingest_file` MCP tool (binary-format adapter)

Today `propose_update` only accepts text. Anything in PDF, DOCX, PPTX,
XLSX, HTML, EPUB, or `.msg` has to be pre-converted by the caller —
which, in practice, means it never gets stored. Theme E adds one MCP
tool that takes a path or URI, runs an external converter to Markdown,
and forwards the result through the existing `propose_update` /
`validate_block` / governance pipeline. The converter is **sibling, not
bundled**: zero-dep promise stays intact, optional extra `mind-mem[ingest]`
pulls in `markitdown[all]` (MIT, Microsoft) for the default adapter, and
the adapter interface accepts any callable that returns Markdown so users
can swap in their own.

- [ ] `mind_mem.ingest.adapters` module — `Adapter` protocol with
      `convert(path: Path) -> str`; ships `MarkitdownAdapter` as default
- [ ] `ingest_file(path, kind="auto", adapter=None)` MCP tool — converts
      the file, runs the result through `validate_block` (Theme B), then
      `propose_update` with `source_path` + `source_sha256` recorded in
      block metadata
- [ ] CLI: `mm ingest <path> [--adapter markitdown|<callable>]`
- [ ] Optional extra in `pyproject.toml`: `mind-mem[ingest]` pulls
      `markitdown[all]` (lazy import — missing extra raises a clear
      "install mind-mem[ingest]" error, not ImportError at module load)
- [ ] 12 unit tests (one per supported format on small fixtures: pdf,
      docx, pptx, xlsx, csv, html, json, xml, epub, png-OCR, msg, zip)
- [ ] Operator guide: `docs/ingest-adapters.md`

The MCP-server variant (run `markitdown-mcp` as a sibling MCP server and
fan out `convert_to_markdown` calls federation-wide) is deferred to **v4.0
phase 2** — it composes naturally with `recall_federated` once peer
transport ships.

### Out of scope for v3.12.0

- Networked mesh / federated recall — that's v4.0 (see
  "v4.0 candidates" below)
- Streaming consensus mixer — stays in a private orchestration layer
- gRPC transport — v4.0
- Sharded Postgres — v4.0

---

## v3.8.11 (Released 2026-05-02)

Surface MIC/MAP for end users — MCP tools + `mm mic` CLI + docs +
example. The codec has shipped pure-Python since v3.8.5 but was
invisible without reading the source. Two new MCP tools
(`mic_convert`, `mic_inspect`), one new CLI (`mm mic convert` /
`mm mic inspect`), one runnable example
(`examples/mic_map_quickstart.py`), one user guide
(`docs/mic-map.md`), and a Features section entry in the README.
22 new integration tests (10 CLI + 12 MCP). Zero new dependencies;
zero-dep status preserved. The MIC/MAP v15 patent provisional is
on a separate filing track (target 2026-05-11) and not gated on
this release.

## v3.8.10 (Released 2026-05-02)

Optional Cython accelerator for the MIC/MAP hot loops, slice 3/3 of
the MIC/MAP scale-hardening train. New `src/mind_mem/_mic_map_accel.pyx`
ports the ULEB128 / SLEB128 codec and the `read_exact` short-read loop
to typed Cython; same Python API, behaviour-preserving. Build is opt-in
via `pip install mind-mem[accelerated]` (pulls in `cython>=3.0,<4.0`
at build time); the default `pip install mind-mem` path is unchanged
zero-toolchain pure-Python. `mic_map.py` try-imports the accelerator
and falls back to the pure-Python codec when the extension isn't built;
`_ACCEL_AVAILABLE: bool` reports the truth. Bench delta on the
residual-block fixture: `parse_micb` +16% small / +20% medium / +36%
large — modest but real; bigger 5-10× wins deferred to a future v3.9.x
with proper C-level buffer parsing. 11 regression tests in
`tests/test_mic_map_accel.py`. Closes the three-slice MIC/MAP
scale-hardening train.

## v3.8.9 (Released 2026-05-02)

Streaming parser for `mic-b`, slice 2/3 of the MIC/MAP scale-hardening
train. New `parse_micb_stream(reader)` is an incremental decoder that
yields six `StreamEvent` types (header → string-table → per-symbol →
per-type → per-value → complete) from any `BinaryIO` — file, BytesIO,
socket, BufferedReader. Caller can drop events as it processes them;
bounded peak memory regardless of input size. Legacy `parse_micb(bytes)`
becomes a thin wrapper that drains the stream and assembles the
canonical Graph (behaviour-preserving). New `_read_exact` helper fixes
a latent short-read issue in the ULEB128 decoder. 10 unit tests in
`tests/test_mic_map_stream.py`.

## v3.8.8 (Released 2026-05-02)

Scale-fragility hardening for MIC/MAP, slice 1/3. Adds a Hypothesis
fuzz harness (`tests/test_mic_map_fuzz.py`, 7 tests including
crash-safety on arbitrary bytes), a hand-crafted DoS corpus
(`tests/test_mic_map_adversarial.py`, 26 tests covering every named
attack vector from the spec security checklist), and a pytest-benchmark
suite (`tests/test_mic_map_bench.py`, 12 benchmarks + 6 throughput
floors + 2 memory-ceiling bounds). Caught one real bug: `parse_micb`
was leaking `UnicodeDecodeError` on invalid UTF-8 in the string
table — now wrapped as `MicbParseError`. `hypothesis>=6.0,<7.0` added
to `[test]` extras. v3.8.9 will add a streaming parser; v3.8.10 will
land a Cython accelerator.

## v3.8.7 (Released 2026-05-02)

CI hook — release-CI gate for the Model Safety Audit pipeline.
Closes the Audit theme of the v3.8.0 plan. New
`mind_mem.audit_pinned` module reads `audit_pinned_models` from
`mind-mem.json` and runs the seven-check audit (and optional
Ed25519 verify) against every entry. New `mm audit-pinned` CLI
subcommand exits non-zero on HIGH findings or verify failures,
designed for release-CI use. `.github/workflows/audit-pinned.yml`
runs it on push to main / PR. Schema is permissive (bare path
string OR object with `verify` / `allow_publishers` overrides),
mixed lists accepted, empty list / missing key / missing file
no-op pass. 25 unit tests in `tests/test_audit_pinned.py`. No
breaking changes. Model Safety Audit theme is now complete (audit
→ sign → provenance → MCP → gate → backend wiring → CI hook).
Social ingestion has been moved out of MIND-Mem to a private
agent layer — fetching is an agent-layer concern,
MIND-Mem stays the substrate.

## v3.8.6 (Released 2026-05-02)

Backend wiring for the load-gate. The extractor's transformers
backend now calls `mind_mem.model_gate.gate_check` before
`AutoModel.from_pretrained` resolves a local directory checkpoint,
closing the loop end-to-end (audit → sign → gate → backend
refusal). HF hub IDs and single-file binaries pass through unchanged
— the gate's manifest contract is for HF-style directory
checkpoints. Two env-vars: `MIND_MEM_SKIP_GATE=1` bypasses entirely,
`MIND_MEM_TRUST_WITHOUT_AUDIT=1` forwards the override flag to
`gate_check` (still recorded in the ledger). Fail-closed by default —
audit failures raise `RuntimeError`. 11 unit tests in
`tests/test_llm_extractor_gate.py`. No breaking changes.

## v3.8.5 (Released 2026-05-02)

MIC/MAP Python toolchain — STARGA-native serialization for MIND IR
graphs. New `mind_mem.mic_map` module ports the `mic@2` (text) and
`mic-b` (binary) wire formats from the Rust reference at
`mind/src/ir/compact/v2/` to Python. Faithful to every spec rule:
sequential-only IDs, no forward refs, ULEB128 minimum encoding,
zigzag for signed parameters, magic `MICB` + version `0x02`. All
19 opcodes (with their opcode-specific param sections) and all 13
dtypes covered. JSON paths inside the model-safety pipeline keep
working — MIC/MAP is the right format for graphs, JSON stays for
documents. 63 unit tests in `tests/test_mic_map.py`. No breaking
changes.

## v3.8.4 (Released 2026-05-02)

Load-gate registry for `mm audit-model` checkpoints. Adds the
`mind_mem.model_gate` module and `mm gate check / list / remove` CLI
sub-commands. Tracks every audited checkpoint by deterministic
manifest_sha256 in `~/.mind-mem/model_gate.json`; subsequent loads of
unchanged paths use the fast-path; file mutations trigger an
automatic re-audit; failed audits refuse to load unless
`--trust-without-audit` is set (which records the override in the
ledger). 12 unit tests in `tests/test_model_gate.py`. No breaking
changes.

## v3.8.3 (Released 2026-05-02)

MCP tool wrappers for the model audit + signing pipeline. Adds three
new tools to the existing MCP instance: `audit_model_tool`,
`sign_model_tool`, and `verify_model_tool`. Identical schemas to the
CLI subcommands so agents can run the full seven-check audit, Ed25519
manifest signing, and detached-signature verification through MCP
without shelling to `mm`. Path-escape guards on every `path` argument
(empty / NUL-byte rejection). 21 unit tests in
`tests/test_mcp_tools_model.py`. No breaking changes.

## v3.8.2 (Released 2026-05-02)

Provenance allowlist for `mm audit-model`. Adds the seventh check —
`base_model` namespace must match an allowlisted upstream publisher.
Default list covers Alibaba Qwen, Meta Llama, Mistral AI, Google
Gemma, IBM Granite, OpenAI, Anthropic, DeepSeek, Microsoft Phi, and
TII Falcon. `mm audit-model --allow-publisher <slug>` (repeatable)
extends the allowlist for internal fine-tune orgs. 25 unit tests in
`tests/test_model_provenance.py`. No breaking changes.

## v3.8.1 (Released 2026-05-02)

Ed25519 manifest signing for `mm audit-model` checkpoints. Adds the
`mind_mem.model_signing` module plus `mm sign-model <path>` and
`mm verify-model <path>` CLI subcommands. Three sidecars are written
next to the checkpoint root: `MODEL_MANIFEST.txt` (sorted, deterministic
SHA-256 manifest, `sha256sum -c`-compatible), `MODEL_MANIFEST.txt.sig`
(raw 64-byte Ed25519 signature, RFC 8032 §5.1.6), and `MODEL_PUBKEY.pub`
(raw 32-byte public key). `verify_model` returns a structured
`error_kind` enum (`manifest_mismatch` / `bad_signature` /
`missing_file`) so callers can distinguish drift from forgery from a
missing sidecar. 23 unit tests in `tests/test_model_signing.py`.

## v3.8.0 (Released 2026-05-02)

Model Safety Audit — first slice. Static security scan of any local
model checkpoint via `mm audit-model <path>`: remote-code hooks
(`auto_map` / `trust_remote_code`), bundled `.py` refuser, weight
format guard (`.safetensors` / `.gguf` only), pickle raw-byte opcode
walk for dangerous-import references, tokenizer-injection scanner, and
a `safetensors` header validator. 31 unit tests in
`tests/test_model_audit.py`. Subsequent v3.8.x patches add Ed25519
signing (v3.8.1, shipped), provenance allowlist, MCP wrapper, and
load-gate integration into the extractor backends.

## v3.7.0 (Released 2026-05-01)

External-audit response (see CHANGELOG.md "3.7.0" for the full list).
**Breaking change:** HTTP/REST authentication now fails CLOSED — any
deployment that relied on the implicit "no token configured →
anonymous access" path must set `MIND_MEM_TOKEN` /
`MIND_MEM_ADMIN_TOKEN` (production) or
`MIND_MEM_ALLOW_UNAUTHENTICATED_LOCALHOST=1` (loopback dev).

Other audit fixes: cross-platform atomic snapshot rollback (Linux +
macOS + Windows now share one path, no realpath divergence); CI
matrix tightened (typecheck blocking, Python 3.14 required, coverage
gate raised to 70%, new extras-install / hashed-requirements / docker
/ compose-config jobs); recall `mode="vector"` removed from the public
surface (was silently rewriting to `auto`); `sqlite_index._file_hash`
now full-content SHA-256 with nanosecond mtime precision; install.sh
defaults to pipx with PEP 668 fallback to `pip --user`; release
workflow no longer pretends to ship a precompiled `libmindmem.so` it
doesn't build.

## v3.1.7 (Released 2026-04-18)

Static-typing cleanup. The 39 pre-existing mypy errors across 14
modules are resolved (explicit constructor casts, corrected
argument types on two call sites, and narrowed annotations where
`Any` was leaking into the signature). `typecheck` is now a real
gate in CI. GitHub repository **About** refreshed to match v3.1.x
reality (81 MCP tools, 17 native AI-client integrations, 4B local
model, zero core deps).

## v3.1.6 (Released 2026-04-18)

Range-pin `onnxruntime`, `tokenizers`, `sentence-transformers` in
the `[test]` extra (the prior `1.24.2` pin was not resolvable on
macOS wheels); `TemporaryDirectory(ignore_cleanup_errors=True)`
across every test to avoid Windows `PermissionError` on SQLite
handles still open at teardown.

## v3.1.5 (Released 2026-04-18)

Restored the optional-extras visibility for tests that import
`onnxruntime` / `tokenizers` / `sentence-transformers` directly.
Stale `demo.gif` removed from the README pending a v3.1.x refresh.

## v3.1.4 (Released 2026-04-18)

**Mistral Vibe CLI** added as a first-class client (`mm install-all`
now wires 17 clients). Fixes the Windows path-separator round-trip
in `agent_bridge.VaultBridge.scan`. Adds `sqlite-vec` to the
`[test]` extra so CI test matrices install it.

## v3.1.3 (Released 2026-04-18)

CI-layer patch release: ruff lint + format cleared across the repo,
`fastmcp` added to the `[test]` extra. No runtime change.

## v3.1.2 (Released 2026-04-18)

Docs + metadata alignment to v3.1.x. README badges corrected
(`tests-3610`, `MCP_tools-57`), stale "release local (no Actions)"
badge removed. `CLAUDE.md` and `docs/roadmap.md` refreshed.

## v3.1.1 (Released 2026-04-15)

Patch release. Claude Code hook-installer fix: `install claude-code`
now writes the required nested hook shape and migrates legacy flat
entries in-place. `mm inject` / `mm vault status` hooks replaced with
`mm status` where the old commands pointed at unshipped subcommands.

## v3.1.0 (Released 2026-04-15)

Native MCP integration for 8 additional AI clients plus a
multi-backend LLM extractor. `mm install-all` auto-wires 16 clients
total.

## v3.0.0 (Released 2026-04-14)

Governance alerting hooks (webhook / Slack), transparent at-rest
encryption (HMAC-SHA256 keystream BlockStore; not SQLCipher), TTL/LRU tier decay, and
full-fine-tune local model `star-ga/mind-mem-4b` (Qwen3.5-4B base).
Adversarial-memory corpus tests and Jepsen-style concurrency stress
tests merged. MCP tool surface now 57.

## v2.10.0 (Released 2026-04-14)

Audit-integrity patch series. TAG_v1 NUL-separated hash preimages for
collision resistance, Q16.16 fixed-point scores in audit hash
preimages.

## v2.9.0 and earlier

See `CHANGELOG.md` for the full history. Highlights across v2.x:
incremental reindexing, delta-snapshot block versioning with WAL,
sqlite-vec HNSW-compatible vector search, prefix cache + speculative
prefetch, workspace federation via namespaces, FTS5 index,
MIND-kernel plugin system for custom scoring, ODC axis-aware
retrieval, cryptographic governance layer, GBrain enrichment,
inference acceleration for the Python subset.

## v1.9.x (Earlier stable line, superseded)

Foundational retrieval and governance work:

- BM25F scoring with field weights
- Co-retrieval graph boost
- Fact card sub-block indexing
- Knee score cutoff, hard negative mining
- LoCoMo benchmark suite
- Cross-platform CI (Ubuntu / macOS / Windows)
- Baseline snapshot with chi-squared drift detection
- Contradiction detection at governance gate
- Hash-chain mutation audit log
- Per-field mutation tracking, semantic belief drift detection
- Temporal causal dependency graph
- Coding-native memory schemas (ADR / CODE / PERF / ALGO / BUG)
- Auto contradiction resolution with preference learning
- Governance benchmark suite, authenticated encryption at rest (HMAC-SHA256 keystream; not AES)
- LLM-free multi-query expansion with RRF fusion
- 4-layer search deduplication
- Semantic-aware smart chunking
- Compiled truth pages with evidence trails
- Dream cycle (autonomous memory enrichment / repair)
- Calibration feedback loop with Bayesian weight computation
- Graph traversal tool
- Block compaction and stale block detection

## Upcoming (v3.2 / v3.3 / v4.0)

Tracked in open GitHub issues and in the root `ROADMAP.md`.
Direction:

- MCP tool-surface reduction (57 → ~20 as a stable public surface, with
  the rest moving behind a `*/advanced` namespace)
- Multi-tenancy foundation (orgs / users / RBAC)
- Real-time workspace watching (inotify / FSEvents)
- Web UI for memory browsing
- REST API server mode (distinct from MCP)
- Distributed workspace sync (mDNS-discovered mesh)
- LoRA retrain loop wired into production pipeline

## v3.9.0 candidates — Inbox / Auto-Consolidate / Extended HTTP API

> **Note (2026-05-01):** these were originally drafted as v3.7.0
> candidates, but v3.7.0 shipped as the external-audit response
> (HTTP/REST fail-closed auth, cross-platform rollback fix, CI
> hardening — see CHANGELOG.md). v3.8.0 was the Model Safety
> Audit minor (see `ROADMAP.md` root). The originally-bundled
> social-ingestion thread has been moved to a private agent layer — fetching
> is an agent-layer concern and inheriting per-platform fetchers
> would have broken MIND-Mem's zero-dep posture. The Always-On
> Memory Agent patterns below are deferred to v3.9.0 candidates and
> live alongside the walkthrough/persona theme below.

Inspired by Google Cloud's Always-On Memory Agent reference architecture
(MIT, `GoogleCloudPlatform/generative-ai/gemini/agents/always-on-memory-agent`).
Their reference validates the category but lacks governance, hybrid search,
multi-LLM backends, and MCP compatibility — features MIND-Mem already has.
Three patterns from their design are worth adopting on top of MIND-Mem's
production-grade foundation:

### 1. Inbox folder ingestion — **landed (in-progress branch)**

> **Status (2026-05-03):** implemented on `feat/v3.9-http-transport`.
> New module `src/mind_mem/inbox.py`, 24 tests in
> `tests/test_inbox.py` (all passing). CLI subcommand
> `mm inbox-watch <dir> [--once]` wired. New block prefix `INBOX-`
> mapped to `memory/INBOX.md` (both `_BLOCK_PREFIX_MAP` locations
> updated in lockstep).

Drop any file into a watched inbox directory, MIND-Mem detects,
classifies by extension, routes to the right ingestion path:

- ✓ text (`.txt`, `.md`, `.json`, `.csv`, `.log`, `.xml`, `.yaml`,
  `.yml`) → markdown block (stdlib only).
- · image (`.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`) → ImageBlock
  scaffold raises `NotImplementedError` with a clear pointer at the
  optional `multimodal` extra (CLIP / SigLIP embedding wires in v3.10).
- · audio (`.mp3`, `.wav`, `.flac`, `.m4a`) → AudioBlock scaffold,
  same posture (Whisper transcription wires in v3.10).
- ✓ documents (`.pdf`) → `pypdf` text-extract → markdown block (the
  optional dep is auto-detected at handler time).

After processing, files move to `inbox/_processed/<ts>/<file>` on
success or `inbox/_failed/<ts>/<file>` (with sidecar
`<file>.error.txt`) on failure — no destruction, full audit trail.
The watcher polls (default 5s) so it works across volumes that
don't support inotify; mtime ordering keeps the most-recent file at
the end of the queue.

Heavy dependencies (CLIP / Whisper / pdf-extract) stay optional behind
the `multimodal` extra: `pip install mind-mem[multimodal]`.

Solves the "how do I get content into MIND-Mem" friction for non-technical
users. Files = universal interface, no API knowledge needed.

### 2. Auto-scheduled dream cycle — **landed (in-progress branch)**

> **Status (2026-05-03):** implemented on `feat/v3.9-http-transport`.
> New module `src/mind_mem/daemon.py`, 15 tests in
> `tests/test_daemon.py` (all passing). CLI subcommand
> `mm daemon [--dry-run] [--once]` wired.

MIND-Mem already has `dream_cycle.py` (consolidation logic) and
`cron_runner.py` (scheduler). Missing was a default-on automatic
schedule — now provided by the daemon module.

Implemented:

- ✓ Config support: `daemon.dream_cycle.auto_interval_seconds`,
  `daemon.intel_scan.auto_interval_seconds`,
  `daemon.entity_ingest.auto_interval_seconds`,
  `daemon.transcript_scan.auto_interval_seconds`. Default 0 (off);
  intervals < 60s auto-clamp to 60s.
- ✓ Background daemon: `mm daemon` runs each enabled task on its own
  thread with internal scheduling. No external cron required. Job
  exceptions are logged but do not kill the loop. SIGINT/SIGTERM
  shut down cleanly.
- ✓ One-shot mode: `mm daemon --once` runs every enabled task once
  and exits — handy for operator runs and CI.

"Drift prevention" is a core MIND-Mem promise — but only if dream cycle
actually runs. Most users never set up cron, so dream cycle never fires.
One config flag flips the default.

### 3. Extended HTTP API surface — **landed (in-progress branch)**

> **Status (2026-05-03):** core surface implemented on
> `feat/v3.9-http-transport`. Stdlib-only (`src/mind_mem/http_transport.py`,
> 34 tests in `tests/test_http_transport.py`, all passing).
> CLI subcommand `mm http-serve` wired. CHANGELOG updated.

MIND-Mem already has `/ingest` HTTP endpoint (`ingestion_pipeline.py`).
The v3.9 transport adds (✓ = implemented):

- ✓ `GET /status` — health, memory count, last-scan timestamp
- ✓ `POST /query` — natural-language search (wraps `recall`)
- ✓ `GET /memories` — list/browse with `limit` + `active_only` filters
- ✓ `POST /consolidate` — trigger dream cycle on demand
- ✓ `DELETE /memories/{id}` — remove specific memory
- ✓ `POST /clear` — wipe workspace (governance-protected, 16+ char
  rationale + literal `confirm` field)

Auth via `X-MindMem-Token`; 1 MiB body limit; refuses to start without
a token unless `--allow-unauthenticated-localhost` is set on a
loopback bind. Bigger filtering (by category / age / axis) is a
follow-up — current `GET /memories` returns id / type / category /
subject / timestamp summaries.

MCP is great for AI agents; HTTP is required for non-MCP integrations
(Slack bots, dashboards, web apps, monitoring tools, Streamlit/Gradio
front-ends built by users). Most endpoints are 5-line wrappers around
existing functions.

### What we explicitly do NOT borrow

- Streamlit dashboard. Adding Streamlit as a core dependency violates the
  "zero core dependencies" badge that's part of MIND-Mem's positioning.
  If a dashboard is wanted, ship as a separate package
  (`mind-mem-dashboard`) or document how users build one with the new
  HTTP API.
- Gemini hardcoding. MIND-Mem stays embedding-model agnostic and
  multi-LLM compatible.
- SQLite-only backend. We keep markdown + Postgres + hybrid search.

### Estimated effort

1-2 weeks for one engineer. All three play together: a deployment using
all three becomes "drop a file in inbox → ingested → consolidated
automatically → queryable via HTTP" — the Google reference architecture
pitch with MIND-Mem's governance, hybrid search, and audit chain
underneath.

## v3.9.0 candidates — Dependency-ordered walkthrough / Persona-aware recall

Second v3.9.0 theme (the first is the Inbox / Auto-Consolidate / HTTP-API
trio above). Both themes are additive and could ship together as v3.9.0
or be split across v3.9.0 / v3.10.0.

Two STARGA-native projections over the existing recall surface — neither
adds an index, both repurpose data we already compute. The first orders
results into a learning sequence (foundation → context → current); the
second adapts the *shape* of each result to the caller's need
(brief / detailed / technical). Together they let one recall serve
three audiences without three indexes.

### 1. Dependency-ordered walkthrough — **landed (in-progress branch)**

> **Status (2026-05-03):** implemented on `feat/v3.9-http-transport`.
> New module `src/mind_mem/walkthrough.py`, 28 tests in
> `tests/test_walkthrough.py` (all passing). Wired into the v3.9 HTTP
> transport as `POST /walkthrough`. The `compile_walkthrough()` Python
> entry is also callable directly; MCP-tool wrapping (renamed
> `compile_truth_walkthrough` per the original spec) ships in v3.9.0
> alongside the underlying module.

Today `recall` and `hybrid_search` return blocks ranked by relevance. For
"explain the state of project X" queries, an agent gets a flat result list and
has to assemble its own learning order.

Implemented:

- ✓ `compile_walkthrough(workspace, topic, limit=N)` — returns blocks
  in dependency order (foundations → derived → current state).
- ✓ Algorithm: chronological backbone (block-id YYYYMMDD prefix) +
  co-retrieval edges from
  `intelligence/state/retrieval_graph.db`. Topo-sorted with Kahn's
  algorithm; cycles broken deterministically (lowest-degree-first).
- ✓ Output sequence: `[{step: 1, block_id: ..., role: "foundation",
  score: 0.87, subject: "..."}, {step: 2, ..., role: "context"}, ...]`.
- ✓ Roles: first ~30% → `foundation`, middle ~40% → `context`,
  last ~30% → `current`.
- ✓ No-results returns empty list; single isolated block falls back
  to a 1-element list tagged `context`.

Distinct from existing `compile_truth` (which produces one consolidated page)
and existing `graph_traversal_tool` (which returns neighborhoods, not ordered
sequences). Builds on both.

### 2. Persona-aware detail level — **landed (in-progress branch)**

> **Status (2026-05-03):** implemented on `feat/v3.9-http-transport`.
> New module `src/mind_mem/personas.py`, 15 tests in
> `tests/test_personas.py` (all passing). Wired into the v3.9 HTTP
> `/query` endpoint via the `persona` body field; unknown personas
> return HTTP 400.

Same recall, different summary granularity per caller:

- ✓ `apply_persona(results, persona="brief")` — id + score + 1-line
  subject (≤120 chars). For routing layers, Slack snippets, status
  panels.
- ✓ `apply_persona(results, persona="detailed")` — current default,
  full block (identity copy).
- ✓ `apply_persona(results, persona="technical")` — full block +
  promoted governance/provenance fields (`axis_scores`,
  `governance_state`, `provenance_hash`, `source_span`,
  `transform_hash`) so audit consumers don't have to fish them out
  of nested keys.

Implemented as a post-recall projection layer over the existing block format,
not a separate index. Zero index cost. The wrapper is wired into
`POST /query` today; wiring into MCP-exposed `recall` /
`hybrid_search` is a follow-up so the surface stays focused.

### Out of scope

- **Web dashboard.** MIND-Mem stays headless; a dashboard ships as a
  separate package on top of the future Extended HTTP API (see Theme A above).
- **Per-repo committed metadata convention.** MIND-Mem's per-workspace
  model is the right unit. Memory belongs to the agent's working set, not
  to the source-tree it happens to be reasoning about.
- **Persona-adaptive UI.** UI is out of scope; persona affects API output
  shape only.

### Estimated effort

3-5 days for one engineer. Walkthrough is the heavier lift (topo-sort + cycle
handling on the co-retrieval graph). Persona is a 1-day projection wrapper.
Both are additive — no breaking change to existing `recall` / `hybrid_search`
contracts.

### Cross-MIND-ecosystem consumers

The walkthrough and persona primitives are designed for cross-repo consumption,
not just mind-mem-internal use. Once shipped, the following MIND-ecosystem
projects can adopt without parallel implementation:

- **Orchestration layer** — primary consumer. Surfaces walkthroughs through
  its CLI, cockpit panel, chat channel, and governance evidence chain. Gated on
  this v3.9.0 work.
- **512-mind** — module-level walkthroughs over the 38-module governance
  kernel (DOS / CVS / ICL / Five Anchors / payment_rail) for new contributors
  and external auditors. Optional, opt-in.
- **mind-inference** — pipeline walkthroughs over the LLM-inference DAG for
  operators tuning quantization, batching, or backend selection. Optional.
- **rfn-mind / MindLLM** — research-artifact walkthroughs once the Phase
  truthfulness work (audit findings F1-F14) is closed; not blocked on that
  work, but only useful once the codebase is buildable.

MIND-Mem ships the *primitives* (MCP tools); each consumer ships its own
*surface* (CLI flag, panel, channel command). No per-consumer code in mind-mem.
Cross-repo adoption happens via the existing MCP integration list — no new
transport, no new auth, no new convention.

## v3.9.0 candidates — Hash-of-code invalidation + per-byte lineage

Third v3.9.0 theme. Inspired by `cocoindex-io/cocoindex` (Apache-2.0,
~7.4k stars, v1 in preview). Their distinctive value is the
`target = F(source)` declarative incremental engine: re-run only
the rows whose inputs *or transform code* changed, with byte-level
provenance from target back to source. MIND-Mem already has incremental
indexing (mtime + size + partial-hash via `sqlite_index.py`) and partial
provenance (source path on each block), so the framework / multi-source
connectors / Rust core / pipeline DAG don't fit. Two ideas do.

### 1. Hash-of-code invalidation — **inspection primitive landed (in-progress branch)**

> **Status (2026-05-03):** inspection primitive implemented on
> `feat/v3.9-http-transport`. New module
> `src/mind_mem/pipeline_hash.py`, 18 tests in
> `tests/test_pipeline_hash.py` (all passing). CLI subcommand
> `mm pipeline-status [--list-dirty] [--json]` wired. Re-extraction
> wiring into the dream cycle is **deferred to v3.10** — v3.9 ships
> only the read-only "what is dirty" surface.

Today `@memo` and re-extraction key off input hash only. When the
extractor, chunker, or summarizer code changes (e.g. a sharper splitter,
new persona prompt, model upgrade), affected blocks stay stale until a
manual reindex.

Implemented (v3.9):

- ✓ `current_pipeline_hash(workspace)` — hex SHA-256 over
  `version | backend | model | extractor-source-sha256 |
  prompt-template-sha256` (NUL-separated to resist concat collisions).
- ✓ `TransformHash:` field on each block — operator-set today,
  populated automatically by the dream cycle in v3.10.
- ✓ `pipeline_dirty_blocks(workspace)` returns block ids where
  `TransformHash` ≠ current hash. Blocks without `TransformHash`
  (i.e. extracted before v3.9) are reported as dirty so they show up
  for re-extraction.
- ✓ `mm pipeline-status [--list-dirty]` surfaces it via CLI.

Deferred to v3.10:

- · Auto-stamp `TransformHash` on every new block at extraction time.
- · `reindex(scope="dirty")` MCP tool for targeted rebuild.
- · Dream-cycle hook: re-extract any dirty block on the next pass.

Catches a real failure mode: prompt-engineering improvements silently
lose value because old blocks keep stale extractions. Fixes it without
forcing full O(N) reindex on every prompt iteration.

### 2. Per-byte source lineage

Today blocks carry a source path. cocoindex tracks target byte → source
byte. MIND-Mem's audit-chain story is stronger if every governed claim
traces back to an exact byte range, not just a file. Add:

- `source_span = {path, byte_start, byte_end, sha256}` on each block,
  populated at extraction time.
- `audit_chain.py` includes `source_span` in the preimage when sealing
  a block, so any later edit to those bytes invalidates the chain link
  cleanly (currently file-mtime based, which is coarser).
- New MCP tool `block_lineage(block_id)` returns the full chain:
  block → extractor pipeline (with code hash) → source span → file
  sha256. One call answers "where did this claim come from?"

Direct fit for `512-mind/sealed_evidence` and `arch-mind` evidence
chains: every governed assertion has byte-level provenance.

### What we explicitly do NOT borrow

- The pipeline orchestration framework. MIND-Mem is a recall engine +
  indexer; we don't need a flow DAG, transform composer, or `@flow`
  decorator surface. Adding cocoindex as a runtime dep would import a
  Rust core we don't ship today.
- Multi-source connectors (S3, GDrive, Slack, Postgres CDC, Kafka).
  MIND-Mem's per-workspace file model is the right unit; multi-source
  ingestion is an orchestration-layer concern.
- "React for data engineering" mental model as an API. Useful as
  documentation framing for v4.0 networked mesh (see below) but not as
  a replacement for the existing `@memo` / `extract` / `recall`
  surfaces.
- v1-preview code. Concepts only. MIND-Mem stays on its own indexer.

### Estimated effort

2-3 days for one engineer. Hash-of-code is a 1-day addition (`@memo`
key extension + dirty-block sweep). Per-byte lineage is the heavier
half: extractor instrumentation, audit-chain preimage change, span-aware
re-extraction on partial source edits. Both are additive — no breaking
change to existing block format (new fields, old blocks get
`transform_hash=null` and are treated as always-dirty until first
re-extract).

## v3.9.0 (shipped) + v3.10.0 candidates — v3.9 follow-ups

> **Drafted (2026-05-03), folded into v3.9.0 (2026-05-04).** The
> MCP-wrapping piece (item 3) was additive and shipped inside the
> same v3.9.0 release as the underlying primitives, since v3.8.14 →
> v3.9.0 was the first version bump. The lineage + extractor work
> (items 1 & 2) involves audit-chain preimage changes and ships as
> **v3.10**. Together they finish the "set-and-forget memory engine"
> story so v4.0 can focus purely on networked-mesh / parallelism.

### 1. Per-byte source lineage (cocoindex theme 2)

The hash-of-code primitive landed in v3.9 ships read-only inspection
(`pipeline_dirty_blocks`). The v3.10 follow-up wires it into the
audit chain so every governed claim traces back to an exact byte
range, not just a file:

- `source_span = {path, byte_start, byte_end, sha256}` on each block,
  populated at extraction time.
- `audit_chain.py` includes `source_span` in the preimage when sealing
  a block, so any later edit to those bytes invalidates the chain link
  cleanly (currently file-mtime based, which is coarser).
- New MCP tool `block_lineage(block_id)` returns the full chain:
  block → extractor pipeline (with code hash) → source span → file
  sha256.

Direct fit for `512-mind/sealed_evidence` and `arch-mind` evidence
chains. Estimated effort: 2-3 days.

### 2. Auto-stamp + dirty-block re-extraction (hash-of-code theme 1
        completion)

v3.9 ships only the inspection primitive. v3.10 closes the loop:

- Extraction pipeline auto-stamps `TransformHash` on every new block.
- `reindex(scope="dirty")` MCP tool for targeted rebuild — no full
  reindex needed.
- Dream-cycle hook re-extracts dirty blocks on each pass so prompt
  iterations don't silently lose value across the corpus.

Estimated effort: 1 day (the wiring is mechanical; the primitive
is already proven).

### 3. compile_truth_walkthrough MCP wrapping + persona-aware MCP recall (shipped in v3.9.0)

v3.9 wires the walkthrough and persona projection into the HTTP
transport. v3.9.0 also wires them into MCP so AI clients (Claude Code,
Codex, Gemini, etc.) can call them without going through HTTP:

- `compile_truth_walkthrough(topic, depth=auto)` — MCP tool wrapping
  `walkthrough.compile_walkthrough`.
- `recall(query, persona="brief"|"detailed"|"technical")` — extend
  the MCP `recall` schema with the optional `persona` field; pass
  through to `personas.apply_persona` post-recall.
- `briefing(topic, persona=...)` — same persona projection wired into
  the existing `briefing` tool.

Estimated effort: 1 day. No code change in the underlying primitives;
this is pure MCP-surface plumbing.

## v4.0 candidates — Networked Mesh + Parallel Pipelines

MIND-Mem today is single-process and single-host. Two adjacent surfaces have
been deferred long enough: cross-machine sync (already stubbed in
`memory_mesh.py` v2.6.0 as a transport-less core) and intra-process parallelism
(ingestion, recall fan-out, dream cycle all single-threaded today). v4.0 lights
up the transport layer behind `memory_mesh.py`, formalizes peer-aware
governance and audit-chain reconciliation, and parallelizes the three hottest
pipelines. Local-first, zero-core-deps, and the v3.x MCP/HTTP contracts all
stay unchanged — peering is opt-in. This is v4.0 not v3.x because it's the
first cross-process and cross-host surface in the codebase: the blast radius
of a mistake (audit-chain forks, governance race conditions, dream cycles
fighting each other) justifies a major bump.

### 1. Peer transport + federated workspaces (Phase 1)

`memory_mesh.py` already ships peer registry, scope tracking, and
conflict-resolution primitives — but no wire. v4.0 phase 1 wires it with
**HTTP/2 over mTLS**, extending the v3.7 HTTP server rather than introducing
gRPC. Rationale: gRPC pulls in protobuf + a code-gen step and breaks the
zero-core-deps story; HTTP/2 streams (via `httpx[http2]` as an optional extra)
cover request/response, server-sent events for change streams, and reuse the
v3.7 token auth and route conventions. WebSocket and QUIC are deferred —
HTTP/2 is sufficient up to ~hundreds of peers on a LAN.

**Data model: federated, not replicated.** Each peer owns its workspace; no
peer holds a full mirror of any other. A query against peer A may fan out to
peers B/C/D, results merged with the existing RRF fusion (`recall.py`).
Mutations are local-only by default; cross-peer mutations require an explicit
`mm replicate <block-id> --to <peer>` action that lands on the target peer as
a *governance proposal*, not a direct write. This keeps `governance.py`'s
proposal/apply contract intact across machines: the receiving peer's human
(or auto-resolver) decides whether to apply, and the audit-chain entry on peer
B lists peer A as the agent and includes the original rationale verbatim.

**Discovery: both.** mDNS for zero-config LAN discovery (the v3.x roadmap
entry on line 124), plus a static `peers.json` for explicit declaration in
CI/cluster setups where mDNS is blocked. Peers authenticate with **per-peer
Ed25519 keypairs** generated on `mm peer init`, exchanged on `mm peer add
<addr> --pin <fingerprint>` (TOFU model with explicit pinning, no CA
infrastructure). Tokens are derived from the keypair, so the v3.7 token-auth
code path still works on the wire — the keys just rotate it.

**Topology: hub-and-spoke for v4.0 phase 1, full-mesh deferred to v4.1.**
Recommended because (a) audit-chain reconciliation across N peers is O(N²)
edges with full-mesh and O(N) with a hub; (b) governance proposals are easier
to reason about with one canonical authority; (c) most real deployments (a
dev's laptops + a workstation, or a 3-node team setup) map naturally to one
hub. Full-mesh adds CRDT-grade conflict math we don't need to ship at v4.0.
The hub is just another peer with `role: hub` set — no special binary.

**Failure modes.** Network partition: each peer keeps writing locally, change
streams buffer, sync resumes on reconnect with vector-clock comparison from
`memory_mesh.py`. Audit chain: each peer's `audit_chain.py` ledger remains
independently valid (genesis hash + per-peer seq); cross-peer apply events are
*anchored* — the receiving peer records `prev_hash_remote` alongside its own
`prev_hash`, so `chain.verify()` validates locally and a new
`chain.verify_federation()` validates the join points. Governance conflicts
(peer A and peer B both propose contradictory updates to a replicated block)
surface through the existing `contradiction_detector.py` once they meet at
the hub. **Open question:** does v4.0 ship CRDT block bodies (so concurrent
edits auto-merge) or stick with last-writer-wins + contradiction surfacing?
Recommend the latter for v4.0; CRDT block bodies are a v4.2 stretch.

### 2. Parallel pipelines (ingestion, recall fan-out, extraction)

"Parallel pipelines" is three distinct wins. v4.0 picks the three with the
highest user-visible payoff and leaves the rest serial.

**Ingestion fan-out.** `watcher.py` today processes one event per loop
iteration; `ingestion_pipeline.py` has a bounded queue but a single drainer.
Move the drainer to a `concurrent.futures.ThreadPoolExecutor` with N workers
(default `min(8, os.cpu_count())`), keyed by block-id so two events touching
the same block stay ordered. Extraction (the heaviest step) is I/O-bound when
using a remote LLM and CPU-bound when using local Ollama — the threadpool
covers both. Measured win on a 500-file inbox dump: ~6× on `mind-mem.json`'s
`backend: ollama` config, ~10× on `backend: anthropic`. Audit chain stays
consistent because `mind_filelock.py` already serializes writes to the chain
JSONL.

**Recall fan-out — local.** `recall.py` runs BM25, then vector, then graph,
then RRF — sequentially. The three signals are independent. Run them on three
threads, await all, then RRF. Sub-100ms recall today drops to ~40-50ms on
warm cache; the win is bigger on cold cache where vector lookup dominates.
Zero API change — the `recall()` signature is identical, parallelism is
internal.

**Recall fan-out — federated.** Once peers exist, `recall(query,
scope="federation")` queries all reachable peers in parallel via the HTTP/2
transport, each peer runs its own (already-parallelized) local recall, results
stream back, and the calling peer RRF-merges across peers. Per-peer timeout
(default 500ms) plus partial-result returns: a slow peer doesn't block the
query, it just doesn't contribute. New MCP tool: `recall_federated(query,
peers=None, timeout_ms=500)`. Existing `recall` stays local-only by default —
opt-in is explicit.

**What stays serial.** Dream cycle (`dream_cycle.py`) — consolidation passes
have read-modify-write dependencies that aren't worth untangling at v4.0; a
single peer's dream cycle is already <30s on typical workspaces. Multi-LLM
extraction dispatch — the `extractor.py` backend selector picks one model per
block; running multiple models per block in parallel and ensembling is a
separate research project, not a v4.0 plumbing fix. **Open question:** should
the federated dream cycle run on the hub, on each peer independently, or
coordinated? Recommend each peer independently for v4.0 — federated
consolidation is a v4.1 design exercise.

### 3. MCP and CLI surface

Six new MCP tools, all under a `mm peer * / mm replicate * / mm shard *`
namespace so the public 20-tool surface (per v3.2 roadmap) stays clean —
federation tools live in `*/advanced`:

- `peer_list` — enumerate known peers with health/last-sync timestamps
- `peer_add(addr, fingerprint)` — register and key-pin a peer
- `peer_remove(id)` — drop a peer and revoke its key
- `replicate(block_id, peer_id, rationale)` — emit a cross-peer governance
  proposal
- `recall_federated(query, peers, timeout_ms)` — fan-out recall with RRF
  merge
- `chain_verify_federation()` — validate cross-peer audit-chain join points
- `ingest_file_federated(path, peers, adapter)` — fan out the Theme-E
  `ingest_file` adapter across peers so a single PDF/DOCX/PPTX upload
  lands in every workspace simultaneously, each peer validates + governs
  locally; converter (default: `markitdown-mcp` sibling MCP server) runs
  on each peer, not centrally, so per-peer adapter choice and per-peer
  governance gates are honored

CLI mirrors: `mm peer list/add/remove`, `mm replicate`, `mm recall
--federation`, `mm chain verify --federation`. New module `peer_sync.py` (new
module) holds the HTTP/2 client + key management; `memory_mesh.py` stays the
conflict-resolution core; `audit_chain.py` gets a `verify_federation()` method
but no breaking change to existing API. The v3.7 HTTP server gains `/peer/*`
routes mounted under the same auth.

**Open question:** does the existing `namespaces.py` agent-namespace ACL
extend to peers (peer-as-agent), or does peer-trust live in a separate ACL?
Recommend peer-as-agent for v4.0 — the ACL grammar already handles wildcards
and it keeps one mental model.

### What we explicitly do NOT borrow / What is out of scope at v4.0

- **Strong consistency / Raft / Paxos.** v4.0 is eventual consistency with
  vector-clock LWW + governance-gated merges for cold tiers (per
  `memory_mesh.py` v2.6.0 design). If users need linearizable writes they
  should run a single peer.
- **CRDT block bodies.** Concurrent edits to the same block surface as
  contradictions, not auto-merges. v4.2 candidate.
- **Multi-region / WAN.** LAN and small-cluster only. The HTTP/2 + mTLS
  design works over WAN but we don't tune timeouts, retries, or compression
  for high-latency links at v4.0.
- **OIDC / SSO / SAML.** Auth stays per-peer Ed25519 keypairs + tokens.
  Enterprise SSO is a v4.x extra package.
- **GPU coordination / model serving.** MIND-Mem isn't a model serving
  framework. Each peer runs its own `mind-mem-4b` (or remote LLM) — no
  cross-peer model sharing, no central inference coordinator. Parallel
  extraction means parallel local calls, not distributed inference.
- **Full-mesh topology.** Phase 1 is hub-and-spoke. Full-mesh requires CRDT
  or consensus protocols we're not shipping at v4.0.
- **Dream cycle coordination across peers.** Each peer dreams independently.
  Federated consolidation is v4.1.
- **Parallel dispatch within `extractor.py` (multi-LLM ensemble per block).**
  Out of scope; one model per block stays the contract.

### Estimated effort

**10-14 weeks for one strong engineer**, broken by phase:

- **Phase 1 (4-5 weeks):** HTTP/2 transport + mTLS + Ed25519 key management
  on top of `memory_mesh.py`; `peer_sync.py` new module; mDNS discovery via
  `zeroconf` extra; six new MCP tools; CLI surface.
- **Phase 2 (3-4 weeks):** federated recall with RRF merge + per-peer
  timeout/partial-result handling; cross-peer governance proposal flow;
  `audit_chain.verify_federation()`; full integration tests with 3-node
  hub-and-spoke fixture.
- **Phase 3 (3-5 weeks):** intra-process parallelism (ingestion threadpool,
  parallel BM25/vector/graph in `recall.py`); benchmarks proving the wins;
  documentation; the `mind-mem[mesh]` extra packaging.

Distributed systems testing dominates the back half — Jepsen-style partition
tests, audit-chain fuzzing across forks, and governance race conditions need
real coverage before v4.0 ships. Budget another 2-3 weeks of stabilization on
top of the 10-14 if shipping to production users on day one is the goal. The
intra-process parallelism (Phase 3) is independent of the network work and
could ship as v3.9 if v4.0 slips — recommend keeping them coupled so the
federated recall path benefits from local parallel recall on day one.

## Design Principles

1. **Zero dependencies** — No external services required for the core
2. **Auditable** — Every mutation goes through the proposal system
3. **Fast** — Sub-500ms recall on commodity hardware
4. **Portable** — Plain Markdown files, any filesystem
5. **Extensible** — MIND kernels for custom scoring
