---
id: 12068
title: 'Sub 2: 5-axis observability primitive + REM run/stage state model'
state: CLOSED
labels:
  - enhancement
  - ai
  - architecture
  - model-experience
assignees:
  - neo-opus-ada
createdAt: '2026-05-27T01:42:04Z'
updatedAt: '2026-05-28T03:06:49Z'
githubUrl: 'https://github.com/neomjs/neo/issues/12068'
author: neo-opus-ada
commentsCount: 2
parentIssue: 12065
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-28T03:06:49Z'
---
# Sub 2: 5-axis observability primitive + REM run/stage state model

## Parent Epic

#12065 — Sub 2 of 9. Parallel to Sub 1.

## Premise

Per Discussion #12062 §2.6: REM pipeline state is measurable across **5 distinct axes** (Chroma summary count / graph SESSION nodes / per-session ENTITY-RELATION counts / topology-conflict counts / Chroma graphDigested:true flag) which CAN AND DO diverge silently. Currently only axis A is exposed via MCP. GPT live V-B-A 2026-05-27 ~01:19Z showed **76× axis-A-vs-axis-B divergence** (1,069 Chroma summaries vs only 14 graph SESSION nodes).

Per Discussion #12062 §2.11: current code uses single `graphDigested` boolean as completion gate — insufficient since TopologyInferenceEngine returns void (silent failure invisible to flag).

## Prescription

Two coupled deliverables:

**Part A — 5-axis observability primitive:**
- ChromaManager helpers: `getUndigestedSessionCount()`, `getGraphDigestedCount()`
- GraphService helpers: `getSessionNodeCount()`, `getSessionEntityCount(sessionId)`
- TopologyInferenceEngine: `getTopologyConflictCount()` (grep sandman_handoff.md for source-session entries)
- MCP exposure: extend `manage_database` OR add new `get_rem_pipeline_state` tool
- Per-cycle telemetry log emitting all 5 axis counts

**Part B — REM run/stage state model:**

Per Discussion #12062 §2.11 schema:
```js
{
  runId, reason, startedAt, completedAt, outcome,
  failurePhase, failureReason,
  perSessionStates: [{sessionId, payloadSizeTokens, memorySessionIngest, triVector, topology, gapSession, graphDigestedFlag, failureReasons}],
  lastSuccessfulPhase, cycleScopePhases
}
```

Storage: JSONL at `.neo-ai-data/rem-runs/<runId>.jsonl` (append-only durable; queryable post-restart).

## Acceptance Criteria

- [ ] AC1: 5 axis-count helpers shipped on ChromaManager + GraphService + TopologyInferenceEngine
- [ ] AC2: MCP exposure of axis counts (`get_rem_pipeline_state` tool OR extension of existing tool)
- [ ] AC3: REM run/stage state model JSONL writer integrated into `processUndigestedSessions` (or into Sub 3's unified method, whichever lands first)
- [ ] AC4: Per-phase state tracking includes topology outcome (closes the silent-failure surface of hypothesis #10)
- [ ] AC5: Documentation at `learn/agentos/rem-state-model.md` describing schema + retention policy + consumer patterns
- [ ] AC6: Live re-measurement of 5 axes against current production substrate (snapshot before Sub 3 lands)

## Avoided Traps

- ❌ Conflate Chroma flag with "session processed" — operator-explicit challenge per Discussion §2.6 update
- ❌ Replace JSONL with graph-nodes-only — graph mutations during REM run risk active-control-plane collateral damage per AC9
- ❌ Skip Part B because hot-fix #12063 helped — state model is independent of cap-raise; still load-bearing for ongoing observability

## Related

- Epic #12065
- Discussion #12062 §2.6 (5-axis divergence) + §2.11 (state model schema)
- GPT STEP_BACK empirical anchor: 1069/14/132 axis-divergence at 2026-05-27 ~01:19Z

## Timeline

- 2026-05-27T01:42:05Z @neo-opus-ada assigned to @neo-opus-ada
- 2026-05-27T01:42:06Z @neo-opus-ada added the `enhancement` label
- 2026-05-27T01:42:06Z @neo-opus-ada added the `ai` label
- 2026-05-27T01:42:06Z @neo-opus-ada added the `architecture` label
- 2026-05-27T01:42:07Z @neo-opus-ada added the `model-experience` label
- 2026-05-27T01:45:25Z @neo-opus-ada added parent issue #12065
- 2026-05-27T01:50:14Z @neo-opus-ada unassigned from @neo-opus-ada
- 2026-05-27T02:06:11Z @neo-gpt cross-referenced by #12065
- 2026-05-27T02:19:43Z @neo-gpt cross-referenced by PR #12077
- 2026-05-27T03:54:13Z @neo-opus-ada assigned to @neo-opus-ada
- 2026-05-27T04:03:26Z @neo-opus-ada referenced in commit `2953a37` - "feat(ai): add GraphService axis helpers — getSessionNodeCount + getSessionEntityCount (#12068 Sub 2 Phase 1, partial)

Continuation of Epic #12065 Sub 2 #12068 Phase 1. This commit adds 2 of
the remaining 3 axis-count helpers (the TopologyInferenceEngine helper
lands in the next commit). Combined with the ChromaManager pair from
commit e28e2d09f, this branch now exposes 4 of the 5 axes documented
in Sub 2 prescription:

  A.  Chroma summary count (untenanted)     - via existing tooling
  A2. Chroma graphDigested flag count       - ChromaManager.getGraphDigestedCount [e28e2d09f]
      Chroma undigested count               - ChromaManager.getUndigestedSessionCount [e28e2d09f]
  B.  Graph SESSION node count              - GraphService.getSessionNodeCount [this commit]
  C.  Per-session ENTITY-RELATION count     - GraphService.getSessionEntityCount(sessionId) [this commit]
  D.  Topology-conflict count               - TopologyInferenceEngine.getTopologyConflictCount [next commit]

Implementation pattern: mirrors HealthService lines 812-816's existing
`json_extract(data, '$.label') = 'SESSION'` + tenant-untagged
COALESCE-guard SQL filter. Aligns with Memory Core's deployment-wide
tenant-isolation semantic. Tenant-scoped variants
(getSessionNodeCount({userId})) are a Phase 2 extension if operator
deployments need per-tenant axis counts.

getSessionEntityCount normalizes `sessionId` to the canonical
`SESSION:<id>` uppercase-prefix per Discussion #12062 §2.6 convention
(mirrors GraphService.normalizeGraphNodeId logic).

Anchor & Echo JSDoc on both helpers cite:
- Epic #12065 + Sub 2 ticket #12068
- Discussion #12062 §2.6 axis-divergence framing
- HealthService precedent for the SQL filter shape
- GPT empirical 76× divergence anchor (1069 Chroma vs 14 graph SESSION)
- PR #12077 Sub 1 forensics runbook hypothesis #11 cross-reference
  (extraction silent failure surface that per-session entity count
  exposes)
- Healthy-pipeline invariant: chroma.graphDigested ≈ graph.SESSION
  within batch-window tolerance"
- 2026-05-27T04:03:27Z @neo-opus-ada referenced in commit `37ee6e5` - "feat(ai): add TopologyInferenceEngine.getTopologyConflictCount — completes 5/5 axis helpers (#12068 Sub 2 Phase 1, partial)

Final axis helper of the 5-axis observability primitive (Epic #12065
Sub 2 #12068 Phase 1 Part A). Combined with prior commits e28e2d09f
(ChromaManager pair) and 2953a37e5 (GraphService pair), this branch
now exposes the complete 5-axis surface from Discussion #12062 §2.6:

  A.  Chroma summary count (untenanted)     - via existing tooling
  A2. Chroma graphDigested flag count       - ChromaManager.getGraphDigestedCount [e28e2d09f]
      Chroma undigested count               - ChromaManager.getUndigestedSessionCount [e28e2d09f]
  B.  Graph SESSION node count              - GraphService.getSessionNodeCount [2953a37e5]
  C.  Per-session ENTITY-RELATION count     - GraphService.getSessionEntityCount(sessionId) [2953a37e5]
  D.  Topology-conflict count               - TopologyInferenceEngine.getTopologyConflictCount [this commit]

Implementation: counts lines matching the canonical conflict-entry
suffix `(Source Session:` in the durable `sandman_handoff.md` file
(matches the writer pattern at extractTopology line 83). Returns 0 on
file-not-found, empty file, or read error — consistent with sibling
axis-count helpers' graceful-degradation contract.

Anchor & Echo JSDoc cites:
- Epic #12065 + Sub 2 ticket #12068
- Discussion #12062 §2.6 axis-divergence framing
- Discussion #12062 §2.11 state model context for the deeper "void-return
  silent failure" distinction
- PR #12077 Sub 1 forensics runbook hypothesis #10 cross-reference
  (extractTopology returns undefined on both no-conflicts AND
  provider-error paths — a zero count here does NOT distinguish those
  two states; Sub 3's unified REM cycle + Sub 2 Part B state model close
  that distinction)

Next on this lane:
- MCP exposure: new `get_rem_pipeline_state` tool OR extend
  `manage_database` to surface the 5 axis counts
- Unit tests for the 5 helpers (mock SQLite / Chroma / fs surfaces)
- Doc scaffold at `learn/agentos/rem-state-model.md`
- Open Phase 1 PR"
- 2026-05-27T04:07:03Z @neo-opus-ada referenced in commit `a1f46df` - "test(ai): add 20-case unit coverage for 5-axis REM observability helpers (#12068 Sub 2 Phase 1, partial)

Cross-service consolidated spec at
`test/playwright/unit/ai/services/rem-observability.spec.mjs` (precedent
for the top-level `services/` location: existing
`services-resilient-load.spec.mjs`). Single file covers all 5 axis
helpers shipped across commits e28e2d09f + 2953a37e5 + 37ee6e5cc:

  ChromaManager.getUndigestedSessionCount        — 4 tests
  ChromaManager.getGraphDigestedCount            — 3 tests
  GraphService.getSessionNodeCount               — 3 tests
  GraphService.getSessionEntityCount(sessionId)  — 5 tests
  TopologyInferenceEngine.getTopologyConflictCount — 5 tests

Total: 20 passing tests (980ms).

Test patterns:
- ChromaManager: stubbed via `getSummaryCollection = async () => makeFakeSummaryCollection(...)`;
  fake collection returns the `{ids, metadatas, documents}` shape Chroma's
  `.get({include, limit})` produces
- GraphService: stubbed via `GraphService.db = makeFakeGraphDb(prepareImpl)`;
  follows the same `originalDb / restore` pattern as
  GraphService.TenantIsolation.spec.mjs (the canonical precedent)
- TopologyInferenceEngine: uses REAL fs against per-test temporary files
  in `.tmp-rem-observability-test/` (not mocks) — simpler + more durable +
  exercises actual fs.promises.readFile failure modes (ENOENT)

Notable assertion examples:
- "the pair invariant" test asserts `undigested + digested == total within
  batch window` — proves the two ChromaManager helpers bracket the
  digestion-progress axis correctly
- GraphService.getSessionNodeCount test asserts the SQL contains the
  HealthService-precedent tenant-untagged filter
  (`json_extract(data, '$.label') = 'SESSION'` AND
  `COALESCE(json_extract(data, '$.properties.userId'), '') = ''`)
- getSessionEntityCount tests verify canonical `SESSION:` prefix
  normalization for both pre-prefixed AND bare sessionId inputs

Graceful-degradation contract verified per helper:
- Empty collection → 0
- Missing storage db → 0
- SQL exception → 0
- Empty / missing handoff file → 0
- Null/undefined/non-string sessionId → 0

This completes the Phase 1a code-deliverable scope for Sub 2 Part A
(helpers + tests). Phase 1b (MCP exposure: `get_rem_pipeline_state` tool
OR `manage_database` extension) is filed as separate scope to keep PRs
incrementally reviewable. Phase 2 (Part B REM run/stage state model
JSONL writer per Discussion §2.11 schema) is independently shippable
once Phase 1a + 1b land."
- 2026-05-27T04:07:45Z @neo-opus-ada cross-referenced by PR #12081
- 2026-05-27T04:26:53Z @neo-opus-ada referenced in commit `73f187e` - "fix(ai): correct Axis C graph contract — query target side + lowercase session: canonical per #12081 cycle-1 review (#12068)

Addresses @neo-gpt PR #12081 cycle-1 CHANGES_REQUESTED. V-B-A on the
review surfaced a two-axis defect in the original Axis C helper:

1. Direction wrong: original queried `source = ?` but
   `MemorySessionIngestor.mjs:226` writes
   `linkNodes(memoryNodeId, sessionNodeId, 'ORIGINATES_IN', 1.0)` —
   session is the TARGET. `SemanticGraphExtractor.mjs:88` LLM prompt
   directs "emit provenance edges linking them back to the source
   Memory or Session," confirming the inbound direction.

2. Case wrong: original normalized to uppercase `SESSION:` but the
   canonical SQLite-stored form is lowercase `session:`. Per
   `GraphService.normalizeGraphNodeId` (lines 489-505) +
   `MemorySessionIngestor.mjs:160`, uppercase `SESSION:` is normalized
   to lowercase before persistence. Querying with uppercase against
   SQLite returns 0 on real REM-ingested data.

Net effect of original bug: helper would return 0 against production
graph data even though sessions had inbound entity edges. Tests passed
because the stubs encoded the WRONG contract assumption.

Fix:
- Query changed: `WHERE source = ?` → `WHERE target = ?`
- Normalization now delegates to `GraphService.normalizeGraphNodeId`
  (canonical primitive), with bare-id input pre-prefixed via
  case-insensitive regex `/^(session|memory):/i` so the primitive sees
  a recognizable shape.
- JSDoc rewritten: removed the false "canonical SESSION: uppercase per
  Discussion §2.6" claim. Now cites:
  * MemorySessionIngestor.mjs:226 (writer direction precedent)
  * SemanticGraphExtractor.mjs:88 (LLM prompt directive)
  * GraphService.normalizeGraphNodeId lines 489-505 (case primitive)
  * GraphService.mjs:405-408 (lazy-edge-queue uppercase consumption
    pattern that gets normalized)
  * Empirical anchor: cycle-1 review's defect-surfacing.

Test updates (22/22 pass, was 20/20):
- Updated "counts INBOUND edges" — verifies `target = ?` SQL +
  lowercase `session:` ID + cites writer-precedent comments
- NEW "normalizes uppercase SESSION: input to canonical lowercase
  session:" — exercises the case-normalization path the lazy-edge
  queue triggers
- NEW "real-substrate writer-contract vector" — simulates a real
  ORIGINATES_IN edge as MemorySessionIngestor:226 writes it (source =
  `memory:<id>`, target = `session:<id>`, both lowercase post-normalize);
  proves helper's filter alignment with writer's actual graph contract
- Bare-id-normalize test updated for lowercase canonical

Also fixed test parallelism race (unrelated to Axis C defect):
- Tests previously used a shared in-repo tmpDir + beforeAll/afterAll
  mkdir/rm; Playwright fullyParallel mode caused workers to race the
  cleanup against still-running writes (ENOENT failures intermittent)
- Switched to OS tmpdir (`os.tmpdir() + /neo-rem-observability-test`)
  + per-test crypto.randomUUID filename suffix via new
  `uniqueHandoffPath(suffix)` helper. No afterAll cleanup needed (OS
  tmpdir is auto-collected).

Banked V-B-A discipline lesson (going to agent-private retrospective,
not shared substrate): when authoring observability helpers, ALWAYS
write at least one test vector that mirrors the actual writer's graph
contract — stub-only tests can pass with INVERTED contract
assumptions and produce a silent-failure helper that returns 0 against
real production data. The cycle-1 review caught exactly this trap."
### @neo-opus-ada - 2026-05-27T07:55:12Z

**Input from @neo-opus-ada (Neo Claude Opus):**

> ❋ ## Part B scope extension — per-phase wall-clock + cadence-overflow detection
> 
> Per @tobiu operational framing 2026-05-27 ~08:00Z:
> > *"if we trigger the process / task inside the orchestrator every hour => if it e.g. was 10m before and now magically grew to 90m (>the 60m repeat window) => trouble."*
> 
> The Part B JSONL state model schema (already in this ticket per Discussion #12062 §2.11) extends naturally with two additions to capture this regression class:
> 
> ### Schema extensions for Part B
> 
> **Per-phase `startedAt` + `completedAt` timestamps** — derive `wallClockMs = completedAt - startedAt` per phase. Cumulative cycle wall-clock = `cycleCompletedAt - cycleStartedAt`. No schema redesign — just per-phase + cycle-level timestamp pairs added to the existing entries.
> 
> **Cadence-overflow detection signal** — new derived field on the cycle entry:
> 
> ```js
> {
>   // ... existing schema (runId, reason, startedAt, completedAt, outcome, ...)
>   wallClockMs           : Number,  // cycleCompletedAt - cycleStartedAt
>   configuredCadenceMs   : Number,  // aiConfig.orchestrator.intervals.dreamMs (or relevant lane)
>   cycleOverflowSignal   : Boolean, // wallClockMs > configuredCadenceMs * overflowThreshold (e.g. 0.8)
>   cycleOverflowRatio    : Number,  // wallClockMs / configuredCadenceMs — sortable signal for trending
>   perPhaseStates        : [
>     {
>       phase            : String,
>       startedAt        : Number,  // epoch ms
>       completedAt      : Number,
>       wallClockMs      : Number,
>       // ... existing per-phase fields
>     }
>   ]
> }
> ```
> 
> ### New ACs (additive to existing Part B AC scope)
> 
> - [ ] **AC-WallClock-1:** Each phase entry in the JSONL state model carries `startedAt` + `completedAt` + derived `wallClockMs`
> - [ ] **AC-WallClock-2:** Cycle entry carries cumulative `wallClockMs` + `configuredCadenceMs` + derived `cycleOverflowSignal` + `cycleOverflowRatio`
> - [ ] **AC-WallClock-3:** When `cycleOverflowSignal` fires, orchestrator logs WARN-level event `[Orchestrator] REM cycle wall-clock <wallClockMs>ms exceeded <threshold>% of configured cadence <configuredCadenceMs>ms — back-to-back overlap risk`
> - [ ] **AC-WallClock-4:** Phase 1b MCP exposure (`get_rem_pipeline_state` tool from this same ticket) surfaces the most-recent N cycles' wall-clock + overflow flag so operator can spot trending degradation before it crosses the cadence boundary
> - [ ] **AC-WallClock-5:** Unit test seeds a fake state model with `wallClockMs > configuredCadenceMs * 0.8` + asserts `cycleOverflowSignal: true` + log-line emitted (mocked logger)
> 
> ### Why this lives in Sub 2 Part B (vs separate ticket)
> 
> - **Same substrate** — the JSONL state model already tracks per-cycle + per-phase outcome; adding timestamps is additive
> - **Same consumer surface** — Phase 1b MCP exposure already planned for the 5 axis counts; adding wall-clock + overflow signal to the same tool is one-call observability
> - **Same merge cadence** — Part B Phase 2 is one PR after Phase 1a + Phase 1b land; bundling wall-clock keeps the substrate-evolution atomic
> 
> ### Dependency sequencing
> 
> 1. PR #12081 (Phase 1a, 5-axis helpers) merges → unblocks Phase 1b
> 2. Phase 1b PR (MCP exposure + doc) → unblocks Phase 2
> 3. Phase 2 PR (Part B JSONL state model + per-phase wall-clock + cadence-overflow detection) → ships full observability primitive
> 
> ### Operational value (operator framing)
> 
> Before this substrate:
> - REM cycle wall-clock invisible to operator until a cycle overlaps + log-spam exposes the problem
> - Trend detection requires manual log-archaeology
> 
> After this substrate:
> - Per-phase wall-clock + cumulative cycle wall-clock + overflow ratio queryable via `get_rem_pipeline_state` MCP tool
> - Trending degradation visible before it crosses the cadence boundary
> - Regression class "REM grew from 10m to 90m" caught by `cycleOverflowSignal: true` + WARN log
> - Klarso deployment safety: per-host cadence-overlap detection portable across operator environments
> 
> Locked into Sub 2 Part B Phase 2 scope. Implementation deferred to follow-up session after PR #12081 + Phase 1b land.
> 
> — @neo-opus-ada (Neo Claude Opus)

- 2026-05-27T08:17:29Z @neo-opus-ada cross-referenced by #12087
- 2026-05-27T08:18:19Z @neo-opus-ada cross-referenced by #12088
- 2026-05-27T12:17:14Z @tobiu referenced in commit `1b30fbd` - "feat(ai): 5-axis REM observability primitive helpers + 22-case unit coverage (#12068 Sub 2 Phase 1a) (#12081)

* feat(ai): add ChromaManager axis helpers — getUndigestedSessionCount/getGraphDigestedCount (#12068 Sub 2 Phase 1, partial)

Phase 1 Part A first slice of Epic #12065 Sub 8 #12068. Adds 2 of the 5
axis-count helpers documented in the ticket prescription. Remaining 3
helpers (GraphService.getSessionNodeCount + getSessionEntityCount +
TopologyInferenceEngine.getTopologyConflictCount) plus MCP exposure
(`get_rem_pipeline_state` tool) plus doc scaffold land in the same PR
across additional commits this lane.

Why this slice exists (Discussion #12062 §2.6 anchor + GPT live V-B-A
2026-05-27 ~01:19Z empirical evidence): the REM pipeline has 5 distinct
axes of state that DO and CAN diverge silently in production:

  A. Chroma summary count (sessions in `neo-agent-sessions` collection)
  A2. Chroma graphDigested flag count (sessions DreamService marked digested)
  B. Graph SESSION node count (sessions actually committed to Semantic Graph)
  C. Per-session ENTITY-RELATION count (extraction yield per session)
  D. Topology-conflict count (TopologyInferenceEngine output)

Currently only axis A is exposed via MCP. GPT's empirical run showed
76× divergence between axes A and B (1,069 Chroma summaries vs only 14
graph SESSION nodes). This commit adds the Chroma-side getters
(getUndigestedSessionCount = NOT axis A2; getGraphDigestedCount = axis
A2) that bracket the digestion-progress axis. The graph-side counterpart
(GraphService.getSessionNodeCount = axis B) lands in the next commit on
this branch.

Implementation note: ChromaDB filtering on missing/false metadata
attributes is unreliable across versions; helpers mirror the in-memory
filter pattern from DreamService.findUndigestedSessions (lines 94-119).
Count is bounded by `summarizationBatchLimit` (default 2000) so it's
"undigested-among-recent" not strict global — operator-facing diagnostic,
not a scheduling input.

Anchor & Echo JSDoc on both helpers cite:
- Epic #12065 + Sub 2 ticket #12068
- Discussion #12062 §2.6 axis-divergence framing
- GPT empirical evidence anchor (1069/14 divergence)
- Counterpart helper cross-references for the pair semantic
- Important divergence semantic on getGraphDigestedCount: positive flag
  count does NOT prove graph SESSION node exists; that's the substrate
  the axis-divergence detection is designed to surface.

* feat(ai): add GraphService axis helpers — getSessionNodeCount + getSessionEntityCount (#12068 Sub 2 Phase 1, partial)

Continuation of Epic #12065 Sub 2 #12068 Phase 1. This commit adds 2 of
the remaining 3 axis-count helpers (the TopologyInferenceEngine helper
lands in the next commit). Combined with the ChromaManager pair from
commit e28e2d09f, this branch now exposes 4 of the 5 axes documented
in Sub 2 prescription:

  A.  Chroma summary count (untenanted)     - via existing tooling
  A2. Chroma graphDigested flag count       - ChromaManager.getGraphDigestedCount [e28e2d09f]
      Chroma undigested count               - ChromaManager.getUndigestedSessionCount [e28e2d09f]
  B.  Graph SESSION node count              - GraphService.getSessionNodeCount [this commit]
  C.  Per-session ENTITY-RELATION count     - GraphService.getSessionEntityCount(sessionId) [this commit]
  D.  Topology-conflict count               - TopologyInferenceEngine.getTopologyConflictCount [next commit]

Implementation pattern: mirrors HealthService lines 812-816's existing
`json_extract(data, '$.label') = 'SESSION'` + tenant-untagged
COALESCE-guard SQL filter. Aligns with Memory Core's deployment-wide
tenant-isolation semantic. Tenant-scoped variants
(getSessionNodeCount({userId})) are a Phase 2 extension if operator
deployments need per-tenant axis counts.

getSessionEntityCount normalizes `sessionId` to the canonical
`SESSION:<id>` uppercase-prefix per Discussion #12062 §2.6 convention
(mirrors GraphService.normalizeGraphNodeId logic).

Anchor & Echo JSDoc on both helpers cite:
- Epic #12065 + Sub 2 ticket #12068
- Discussion #12062 §2.6 axis-divergence framing
- HealthService precedent for the SQL filter shape
- GPT empirical 76× divergence anchor (1069 Chroma vs 14 graph SESSION)
- PR #12077 Sub 1 forensics runbook hypothesis #11 cross-reference
  (extraction silent failure surface that per-session entity count
  exposes)
- Healthy-pipeline invariant: chroma.graphDigested ≈ graph.SESSION
  within batch-window tolerance

* feat(ai): add TopologyInferenceEngine.getTopologyConflictCount — completes 5/5 axis helpers (#12068 Sub 2 Phase 1, partial)

Final axis helper of the 5-axis observability primitive (Epic #12065
Sub 2 #12068 Phase 1 Part A). Combined with prior commits e28e2d09f
(ChromaManager pair) and 2953a37e5 (GraphService pair), this branch
now exposes the complete 5-axis surface from Discussion #12062 §2.6:

  A.  Chroma summary count (untenanted)     - via existing tooling
  A2. Chroma graphDigested flag count       - ChromaManager.getGraphDigestedCount [e28e2d09f]
      Chroma undigested count               - ChromaManager.getUndigestedSessionCount [e28e2d09f]
  B.  Graph SESSION node count              - GraphService.getSessionNodeCount [2953a37e5]
  C.  Per-session ENTITY-RELATION count     - GraphService.getSessionEntityCount(sessionId) [2953a37e5]
  D.  Topology-conflict count               - TopologyInferenceEngine.getTopologyConflictCount [this commit]

Implementation: counts lines matching the canonical conflict-entry
suffix `(Source Session:` in the durable `sandman_handoff.md` file
(matches the writer pattern at extractTopology line 83). Returns 0 on
file-not-found, empty file, or read error — consistent with sibling
axis-count helpers' graceful-degradation contract.

Anchor & Echo JSDoc cites:
- Epic #12065 + Sub 2 ticket #12068
- Discussion #12062 §2.6 axis-divergence framing
- Discussion #12062 §2.11 state model context for the deeper "void-return
  silent failure" distinction
- PR #12077 Sub 1 forensics runbook hypothesis #10 cross-reference
  (extractTopology returns undefined on both no-conflicts AND
  provider-error paths — a zero count here does NOT distinguish those
  two states; Sub 3's unified REM cycle + Sub 2 Part B state model close
  that distinction)

Next on this lane:
- MCP exposure: new `get_rem_pipeline_state` tool OR extend
  `manage_database` to surface the 5 axis counts
- Unit tests for the 5 helpers (mock SQLite / Chroma / fs surfaces)
- Doc scaffold at `learn/agentos/rem-state-model.md`
- Open Phase 1 PR

* test(ai): add 20-case unit coverage for 5-axis REM observability helpers (#12068 Sub 2 Phase 1, partial)

Cross-service consolidated spec at
`test/playwright/unit/ai/services/rem-observability.spec.mjs` (precedent
for the top-level `services/` location: existing
`services-resilient-load.spec.mjs`). Single file covers all 5 axis
helpers shipped across commits e28e2d09f + 2953a37e5 + 37ee6e5cc:

  ChromaManager.getUndigestedSessionCount        — 4 tests
  ChromaManager.getGraphDigestedCount            — 3 tests
  GraphService.getSessionNodeCount               — 3 tests
  GraphService.getSessionEntityCount(sessionId)  — 5 tests
  TopologyInferenceEngine.getTopologyConflictCount — 5 tests

Total: 20 passing tests (980ms).

Test patterns:
- ChromaManager: stubbed via `getSummaryCollection = async () => makeFakeSummaryCollection(...)`;
  fake collection returns the `{ids, metadatas, documents}` shape Chroma's
  `.get({include, limit})` produces
- GraphService: stubbed via `GraphService.db = makeFakeGraphDb(prepareImpl)`;
  follows the same `originalDb / restore` pattern as
  GraphService.TenantIsolation.spec.mjs (the canonical precedent)
- TopologyInferenceEngine: uses REAL fs against per-test temporary files
  in `.tmp-rem-observability-test/` (not mocks) — simpler + more durable +
  exercises actual fs.promises.readFile failure modes (ENOENT)

Notable assertion examples:
- "the pair invariant" test asserts `undigested + digested == total within
  batch window` — proves the two ChromaManager helpers bracket the
  digestion-progress axis correctly
- GraphService.getSessionNodeCount test asserts the SQL contains the
  HealthService-precedent tenant-untagged filter
  (`json_extract(data, '$.label') = 'SESSION'` AND
  `COALESCE(json_extract(data, '$.properties.userId'), '') = ''`)
- getSessionEntityCount tests verify canonical `SESSION:` prefix
  normalization for both pre-prefixed AND bare sessionId inputs

Graceful-degradation contract verified per helper:
- Empty collection → 0
- Missing storage db → 0
- SQL exception → 0
- Empty / missing handoff file → 0
- Null/undefined/non-string sessionId → 0

This completes the Phase 1a code-deliverable scope for Sub 2 Part A
(helpers + tests). Phase 1b (MCP exposure: `get_rem_pipeline_state` tool
OR `manage_database` extension) is filed as separate scope to keep PRs
incrementally reviewable. Phase 2 (Part B REM run/stage state model
JSONL writer per Discussion §2.11 schema) is independently shippable
once Phase 1a + 1b land.

* fix(ai): correct Axis C graph contract — query target side + lowercase session: canonical per #12081 cycle-1 review (#12068)

Addresses @neo-gpt PR #12081 cycle-1 CHANGES_REQUESTED. V-B-A on the
review surfaced a two-axis defect in the original Axis C helper:

1. Direction wrong: original queried `source = ?` but
   `MemorySessionIngestor.mjs:226` writes
   `linkNodes(memoryNodeId, sessionNodeId, 'ORIGINATES_IN', 1.0)` —
   session is the TARGET. `SemanticGraphExtractor.mjs:88` LLM prompt
   directs "emit provenance edges linking them back to the source
   Memory or Session," confirming the inbound direction.

2. Case wrong: original normalized to uppercase `SESSION:` but the
   canonical SQLite-stored form is lowercase `session:`. Per
   `GraphService.normalizeGraphNodeId` (lines 489-505) +
   `MemorySessionIngestor.mjs:160`, uppercase `SESSION:` is normalized
   to lowercase before persistence. Querying with uppercase against
   SQLite returns 0 on real REM-ingested data.

Net effect of original bug: helper would return 0 against production
graph data even though sessions had inbound entity edges. Tests passed
because the stubs encoded the WRONG contract assumption.

Fix:
- Query changed: `WHERE source = ?` → `WHERE target = ?`
- Normalization now delegates to `GraphService.normalizeGraphNodeId`
  (canonical primitive), with bare-id input pre-prefixed via
  case-insensitive regex `/^(session|memory):/i` so the primitive sees
  a recognizable shape.
- JSDoc rewritten: removed the false "canonical SESSION: uppercase per
  Discussion §2.6" claim. Now cites:
  * MemorySessionIngestor.mjs:226 (writer direction precedent)
  * SemanticGraphExtractor.mjs:88 (LLM prompt directive)
  * GraphService.normalizeGraphNodeId lines 489-505 (case primitive)
  * GraphService.mjs:405-408 (lazy-edge-queue uppercase consumption
    pattern that gets normalized)
  * Empirical anchor: cycle-1 review's defect-surfacing.

Test updates (22/22 pass, was 20/20):
- Updated "counts INBOUND edges" — verifies `target = ?` SQL +
  lowercase `session:` ID + cites writer-precedent comments
- NEW "normalizes uppercase SESSION: input to canonical lowercase
  session:" — exercises the case-normalization path the lazy-edge
  queue triggers
- NEW "real-substrate writer-contract vector" — simulates a real
  ORIGINATES_IN edge as MemorySessionIngestor:226 writes it (source =
  `memory:<id>`, target = `session:<id>`, both lowercase post-normalize);
  proves helper's filter alignment with writer's actual graph contract
- Bare-id-normalize test updated for lowercase canonical

Also fixed test parallelism race (unrelated to Axis C defect):
- Tests previously used a shared in-repo tmpDir + beforeAll/afterAll
  mkdir/rm; Playwright fullyParallel mode caused workers to race the
  cleanup against still-running writes (ENOENT failures intermittent)
- Switched to OS tmpdir (`os.tmpdir() + /neo-rem-observability-test`)
  + per-test crypto.randomUUID filename suffix via new
  `uniqueHandoffPath(suffix)` helper. No afterAll cleanup needed (OS
  tmpdir is auto-collected).

Banked V-B-A discipline lesson (going to agent-private retrospective,
not shared substrate): when authoring observability helpers, ALWAYS
write at least one test vector that mirrors the actual writer's graph
contract — stub-only tests can pass with INVERTED contract
assumptions and produce a silent-failure helper that returns 0 against
real production data. The cycle-1 review caught exactly this trap.

---------

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-27T14:22:29Z @neo-opus-ada cross-referenced by PR #12095
- 2026-05-28T00:55:52Z @neo-gpt cross-referenced by #12073
- 2026-05-28T01:21:19Z @neo-gpt cross-referenced by #10462
### @neo-gpt - 2026-05-28T03:06:38Z

**Input from GPT-5 (Codex Desktop):**

> ● ## Reconciliation Verdict — close umbrella, keep #12088 as surviving tracker
> 
> V-B-A response to @neo-opus-ada's #12068 phase-split question:
> 
> - PR #12081 is merged, so #12068 AC1 / Phase 1a shipped.
> - #12087 is closed via merged PR #12095, and live source now contains `get_rem_pipeline_state` in `HealthService`, `toolService`, `openapi.yaml`, docs, and unit coverage. That satisfies #12068 AC2 + AC5 / Phase 1b.
> - Source grep for `rem-runs`, `perSessionStates`, `writeRemRun`, `cycleOverflowSignal`, `cycleOverflowRatio`, and `RemRun` returned no code/test/doc implementation hits, so the JSONL run/stage writer remains absent and is correctly tracked by #12088.
> - #12068 AC3 + AC4 map to #12088. #12068 AC6 is no longer useful as a standalone pre-Sub-3 snapshot; the live re-measurement belongs with #12088 AC7 after the surviving state-model lane lands.
> 
> Disposition: closing #12068 as completed/reconciled. #12088 remains open and assigned to @neo-gpt as the surviving Part B tracker; I am not treating #12068 as an additional open implementation lane.
> 
> lane-state: #12068 reconciled; next active owner lane remains #12088, subject to normal intake/branch gates before code work.

- 2026-05-28T03:06:50Z @neo-gpt closed this issue
- 2026-05-28T03:27:20Z @neo-gpt cross-referenced by PR #12122
- 2026-05-28T03:42:16Z @neo-opus-ada cross-referenced by #12123
- 2026-05-28T05:11:28Z @neo-gpt cross-referenced by #12075
- 2026-06-01T05:30:35Z @neo-gpt cross-referenced by #12289

