---
id: 10462
title: 'Graph-integrity audit: verify SESSION→memory completeness post-REM'
state: CLOSED
labels:
  - enhancement
  - ai
  - architecture
assignees:
  - neo-gpt
createdAt: '2026-04-28T08:50:35Z'
updatedAt: '2026-05-28T07:21:10Z'
githubUrl: 'https://github.com/neomjs/neo/issues/10462'
author: neo-opus-ada
commentsCount: 1
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-28T07:21:10Z'
---
# Graph-integrity audit: verify SESSION→memory completeness post-REM

## Context

Surfaced 2026-04-28 during research on a silent-partial-ingestion bug in `DreamService` (sibling ticket). Even after that gate is tightened, the graph today carries an unknown number of historically-orphaned memories from sessions that were marked `graphDigested: true` while some of their `MemorySessionIngestor` upserts had silently failed. There is currently no substrate that can detect or report this divergence.

This ticket proposes a **read-only graph-integrity audit** — a periodic sweep that, for every `SESSION` graph node, queries Chroma for the expected raw-memory count and compares with the graph's `ORIGINATES_IN`-edge count. Mismatches surface in a structured report consumable by humans and downstream agents (Memory Core, Sandman handoff).

## The Problem

The Native Edge Graph and Chroma memory collection are kept in sync by `MemorySessionIngestor`. The synchronization is one-shot per session — once the session is marked digested, no integrity check ever revisits it. Conditions that produce silent drops include:

- Transient SQLite parameter overflow (sibling ticket — was the trigger today)
- Disk-space exhaustion mid-write
- Lock contention from concurrent processes
- Schema-mismatch errors caught by the per-memory `try/catch`
- Future unknown-unknown failures in any upstream service

Without a substrate to detect divergence, these silent drops accumulate. The Memory Core's effective recall fidelity decays, but the system reports green status. Sandman's handoff and Golden Path synthesis run on partial data without indication. The longer the delay between drop and detection, the harder root-cause attribution becomes.

A periodic integrity audit is the substrate-level safety net — it doesn't prevent drops, it surfaces them.

## The Architectural Reality

**Files involved (read-only — this is an audit, not a mutator):**

- `ai/mcp/server/memory-core/services/GraphService.mjs` — read SESSION nodes + `ORIGINATES_IN` edge counts
- `ai/mcp/server/memory-core/services/MemoryService.mjs` (or `StorageRouter.getMemoryCollection()`) — read Chroma memory-collection counts grouped by `sessionId`
- `ai/scripts/` — entry-point script for one-shot audit
- `ai/services.mjs` — SDK aggregation for both reads, with `KB_Config.data.autoSync = false` and `GH_Config.data.syncOnStartup = false` already in place (precedent: `buildScripts/ai/backup.mjs`)

**What the audit emits:**

A structured report (JSON or markdown) with per-session entries:
- `sessionId`, `sessionNodeId`, `chromaSessionId`
- `expectedMemoryCount` (from Chroma `where: {sessionId: X}` count)
- `actualMemoryCount` (from graph `ORIGINATES_IN` edges into `session:X`)
- `divergence` (positive = missing in graph; negative = orphaned in graph)
- `severity` (clean / soft drop ≤5% / hard drop >5%)

**Consumer identity:** primary consumer is the maintainer (operator). Secondary consumer is downstream automation that could trigger lazy back-fill on detected gaps. Tertiary is the swarm — the report could feed into Sandman's golden-path synthesis as a coverage-quality signal.

## The Fix

Net-new substrate. Phased implementation:

**Phase A (this ticket — minimum viable):** standalone `npm run` script.
- New file: `ai/scripts/auditGraphIntegrity.mjs` (or `buildScripts/ai/auditGraphIntegrity.mjs` — pick the directory whose conventions match; `buildScripts/ai/backup.mjs` is the precedent for aggregation-via-`ai/services.mjs`).
- Reads all SESSION nodes from graph, all memory rows from Chroma grouped by `sessionId`, computes divergence per session.
- Writes report to `.neo-ai-data/audits/graph-integrity-<ISO-timestamp>.json` (or `.md` for human-readable).
- Exit code 0 (clean), 1 (soft drops), 2 (hard drops) so CI / cron can branch.
- New `package.json` script: `"ai:audit-integrity": "node ./buildScripts/ai/auditGraphIntegrity.mjs"`.

**Phase B (future ticket):** integrate audit output into Sandman's `synthesizeGoldenPath` so coverage-quality signals appear in `sandman_handoff.md`.

**Phase C (future ticket):** auto-trigger lazy back-fill on detected gaps via `MemorySessionIngestor.ingestSingleRow`.

This ticket is scoped to Phase A.

## Acceptance Criteria

- [ ] `npm run ai:audit-integrity` runs end-to-end on the current Memory Core + graph DB without crashes
- [ ] Report file is written under `.neo-ai-data/audits/` with ISO-timestamped filename
- [ ] Per-session entry includes `sessionId`, `expectedMemoryCount`, `actualMemoryCount`, `divergence`, `severity`
- [ ] Severity classification: `clean` (divergence === 0), `soft` (1-5% drop), `hard` (>5% drop) — thresholds configurable via env or config keys
- [ ] Exit code reflects worst severity: 0 for clean-only, 1 for soft, 2 for hard
- [ ] Read-only: no graph-DB mutations, no Chroma writes, no service-side side-effects
- [ ] Unit test: synthetic fixture with 3 sessions (one clean, one soft, one hard) verifies report shape and exit code
- [ ] Empirical: dry-run on the live Memory Core after the sibling ticket fixes ship; report surfaces any historical silent drops
- [ ] Commit subject ends with `(#TICKET_ID)` per AGENTS.md §3 Gate 1; type=`feat`, scope=`ai`

## Out of Scope

- **Phase B (Sandman integration) and Phase C (auto-backfill).** Filed as future tickets after Phase A's report shape is empirically validated.
- **Cross-tenant scoping.** The audit operates on the current `userId` context. Multi-tenant isolation belongs to the broader #9999 epic.
- **Real-time streaming integrity check.** This is a periodic audit, not a real-time invariant. Streaming integrity would be a separate substrate.
- **Graph→Chroma direction (orphaned graph nodes with no Chroma source).** This ticket is scoped to "missing memories." Chroma rows referenced by graph but absent in Chroma is a different failure pattern (Chroma data loss) — file separately if observed.

## Avoided Traps

- **Embedding the audit in the runSandman finally block.** Rejected. RunSandman is already too long-lived (~10 min); adding a full graph traversal at the tail compounds the time. Standalone script is independently runnable, separable, and operator-controlled.
- **MCP tool surface for the audit at this stage.** Rejected for Phase A. MCP tools imply schema closure (per `MCP output category split` pattern); audit output is exploratory at this point — let the report shape stabilize empirically before pinning a closed contract.
- **Per-memory hash verification (audit each individual memory's `payloadHash`).** Rejected for Phase A. Count divergence is the cheap signal that catches the failure mode this ticket targets. Hash-level audit would catch a different class of corruption (mutated memory content) and is substantially more expensive — file separately if needed.
- **Audit-time auto-correction.** Rejected. Phase A is observation only. Mixing detection with correction in the same script bypasses the operator's review of severity. Phase C is the correct home for auto-correction, after operators trust the detection signal.

## Related

- Detects historical drops from: sibling ticket on DreamService `graphDigested` gate (which prevents future drops but doesn't backfill past ones)
- Trigger source for past drops: sibling ticket on SQLite IN-clause overflow (and any other transient ingestion failure mode)
- Adjacent: #10153 (lazy back-fill — audit could feed it; out of scope for Phase A)
- Adjacent: #10158 (post-ship telemetry + retention policy for Memory/Session graph nodes — natural co-evolution path)
- Adjacent: #10143 (Memory + Session as first-class graph nodes — this ticket audits the integrity of that substrate)

Origin Session ID: `4bb6859b-860f-440d-9055-320e20b0ee22`

Retrieval Hint: `graph-integrity audit SESSION ORIGINATES_IN edge count completeness check session-to-memory divergence orphaned-memory detection`

## Timeline

- 2026-04-28T08:50:36Z @neo-opus-ada added the `enhancement` label
- 2026-04-28T08:50:37Z @neo-opus-ada added the `ai` label
- 2026-04-28T08:50:37Z @neo-opus-ada added the `architecture` label
- 2026-04-28T09:32:03Z @neo-opus-ada cross-referenced by #10467
- 2026-04-28T10:41:52Z @neo-opus-ada cross-referenced by #10469
- 2026-05-03T14:53:43Z @neo-gpt cross-referenced by #10647
- 2026-05-03T15:03:08Z @neo-gpt cross-referenced by #10651
- 2026-05-03T21:19:12Z @neo-gpt cross-referenced by #10666
### @neo-gpt - 2026-05-28T01:21:18Z

**Input from GPT-5.5 (Codex Desktop):**

> ● ## Ticket Intake Revalidation — still valid, but needs current-substrate narrowing before pickup
> 
> Verdict: `needs-narrowing / needs-contract-refresh` before implementation. I am not self-assigning or starting a branch from the April prescription as written.
> 
> Fresh evidence checked on 2026-05-28:
> 
> - No successor PR closes #10462 directly (`gh search prs "10462 OR graph-integrity OR audit-integrity OR rem-observability OR getSessionEntityCount"` returned no direct successor PRs).
> - Later REM observability work under #12068 added `GraphService.getSessionEntityCount(sessionId)` and `test/playwright/unit/ai/services/rem-observability.spec.mjs`, which cover the graph-side per-session entity-count primitive this ticket originally needed to invent.
> - Current KB/source evidence confirms `getSessionEntityCount()` counts inbound edges where the canonical lowercase `session:<id>` node is the target, matching `MemorySessionIngestor`'s `ORIGINATES_IN(Memory → Session)` writer contract.
> - The still-missing surface is the standalone audit/report runner: comparing expected Chroma raw-memory counts against graph `ORIGINATES_IN` counts, emitting `.neo-ai-data/audits/graph-integrity-<timestamp>.json`, and returning severity exit codes.
> 
> Required before implementation:
> 
> 1. Update the implementation plan to consume the existing #12068 helper instead of re-deriving graph edge-count logic.
> 2. Add/confirm a Contract Ledger for the CLI + report surface:
> 
> | Surface | Before | After | Consumer |
> |---|---|---|---|
> | `npm run ai:audit-integrity` | absent | standalone read-only audit runner | operator / CI / future Sandman integration |
> | `.neo-ai-data/audits/graph-integrity-*.json` | absent | timestamped report with `sessionId`, expected/actual counts, divergence, severity | operator + downstream automation |
> | graph-side session count | helper absent in April body | reuse `GraphService.getSessionEntityCount(sessionId)` from #12068 | audit runner |
> | Chroma expected count | unspecified helper boundary | explicit injectable/SDK read path for raw memories grouped by `sessionId` | audit runner tests + live dry run |
> 
> The ticket remains useful, but the branch should not start until the body/comment contract is treated as the current pickup basis. No assignment, no branch, no file edits in this pass.

- 2026-05-28T04:07:21Z @neo-gpt assigned to @neo-gpt
- 2026-05-28T04:14:15Z @neo-gpt cross-referenced by PR #12124
- 2026-05-28T06:57:50Z @neo-gpt referenced in commit `7a25ca5` - "feat(ai): add graph integrity audit runner (#10462)"
- 2026-05-28T06:57:50Z @neo-gpt referenced in commit `6c21758` - "fix(ai): use commander for graph audit CLI (#10462)"
- 2026-05-28T07:21:10Z @tobiu referenced in commit `e5bd1f6` - "feat(ai): add graph integrity audit runner (#10462) (#12124)

* feat(ai): add graph integrity audit runner (#10462)

* fix(ai): use commander for graph audit CLI (#10462)

---------

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-28T07:21:10Z @tobiu closed this issue

