---
id: 10944
title: Remove stale sqlite-vec artifacts from Memory Core healthcheck
state: CLOSED
labels:
  - bug
  - documentation
  - ai
  - refactoring
  - architecture
assignees:
  - neo-gpt
createdAt: '2026-05-08T10:06:00Z'
updatedAt: '2026-05-09T21:15:40Z'
githubUrl: 'https://github.com/neomjs/neo/issues/10944'
author: neo-gpt
commentsCount: 2
parentIssue: 10822
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-08T10:26:42Z'
---
# Remove stale sqlite-vec artifacts from Memory Core healthcheck

## Context

Operator surfaced a healthcheck semantics drift on 2026-05-08: Memory Core now uses **ChromaDB** for memories and session summaries, while **better-sqlite3** backs the Native Edge Graph. The prior sqlite-vec / SQLite-vector alternative was empirically rejected and is no longer a supported path: no HNSW, no skip lists, brute-force vector scans only. There is no backwards-compatibility requirement for that path.

Current `healthcheck` output still exposes confusing SQLite-vector-era artifacts. Empirical snapshot from @neo-gpt current session:

```json
"database": {
  "connection": {
    "connected": true,
    "collections": {
      "memories": {"name": "neo-agent-memory", "exists": true, "count": 293},
      "summaries": {"name": "neo-agent-sessions", "exists": true, "count": 0}
    },
    "engines": {"chroma": true, "sqlite": false}
  }
},
"migration": {"memory": 0, "session": 0, "total": 0, "available": true}
```

That reads as if SQLite is an optional/failed database engine beside Chroma, while another block simultaneously proves the SQLite graph is queryable enough for migration counts. The result is misleading operator and agent diagnostics.

Duplicate sweep:
- #9924 is the closed April two-pillar migration ticket; it established separate Chroma/graph diagnostics before the current sqlite-vec cleanup reality.
- #10773/#10799 shipped SQLite-side embedding-provider observability, then #10804 consolidated provider selection; remaining docs/comments still carry stale SQLite-side embedding language.
- #10822 already records sqlite-vec as an avoided dead end and mandates hard cuts, but it does not yet have a concrete healthcheck cleanup sub-ticket.

## The Problem

The healthcheck payload and docs still encode stale concepts from a dropped design:

1. `database.connection.engines.sqlite` is a permanent false-looking artifact. `HealthService.#checkDatabaseConnections()` initializes `{chroma: false, sqlite: false}` and only ever sets `engines.chroma = true`; it never validates the better-sqlite graph as a peer vector engine.
2. The `providers.embedding` docs/comments say the embedding route is used by “ChromaDB retrieval and SQLite Native Edge Graph operations.” That is now misleading: Chroma owns vector memory/session retrieval; better-sqlite owns graph topology.
3. `config.template.mjs` still says `vectorDimension` is enforced “across all SQLite collections,” which points readers back toward the dropped sqlite-vec mental model.
4. `openapi.yaml` still documents `database.connection.engines.sqlite` as a boolean response field.

This creates exactly the failure mode seen during the wake incident investigation: agents over-interpret `engines.sqlite=false` as a graph-health signal, even though the field is not the source of truth for better-sqlite graph readiness.

## The Architectural Reality

- Memory/session vector storage: ChromaDB collections `neo-agent-memory` and `neo-agent-sessions` via `StorageRouter` / `ChromaManager`.
- Graph storage: better-sqlite3-backed Native Edge Graph via `ai/graph/storage/SQLite.mjs` and `GraphService.db.storage.db`.
- Healthcheck source: `ai/mcp/server/memory-core/services/HealthService.mjs`.
- Healthcheck schema: `ai/mcp/server/memory-core/openapi.yaml`.
- Operator docs: `learn/agentos/MemoryCore.md` and shared deployment docs.
- Parent cleanup epic: #10822, especially the “sqlite-vec dead-end; Chroma stays for vectors; better-sqlite stays for graph” constraint.

## The Fix

Hard-cut stale sqlite-vec / SQLite-vector artifacts from the Memory Core healthcheck contract and docs.

Recommended shape:

1. Replace or remove `database.connection.engines.sqlite`. If graph readiness is needed, expose it under an explicit graph block such as:

```json
"database": {
  "connection": {
    "engines": {"chroma": true},
    "graph": {
      "backend": "better-sqlite3",
      "available": true,
      "path": ".neo-ai-data/sqlite/memory-core-graph.sqlite"
    }
  }
}
```

2. Ensure graph availability, if surfaced, comes from the actual better-sqlite graph handle/path, not from retired vector-engine state.
3. Update `openapi.yaml` to remove or rename the stale `sqlite` engine field.
4. Update `learn/agentos/MemoryCore.md` and relevant config comments to state the current split clearly: ChromaDB for memories/summaries vectors; better-sqlite3 for Native Edge Graph topology.
5. Remove stale references to SQLite-side embedding / sqlite-vec compatibility unless they are explicitly in “rejected/dead-end” history sections.

## Contract Ledger Matrix

| Target Surface | Source of Authority | Proposed Behavior | Fallback / Edge Case | Docs | Evidence |
|---|---|---|---|---|---|
| Memory Core `healthcheck.database.connection` payload | This ticket, #10822, current operator correction | Chroma connectivity reports the Chroma vector substrate for `neo-agent-memory` / `neo-agent-sessions`; graph readiness, if reported, is explicitly named as better-sqlite3 graph readiness, not `sqlite` vector-engine status. | If GraphService is not initialized during cold healthcheck, graph block returns `available: false` with a clear readiness reason; it must not imply sqlite-vec is supported. | `learn/agentos/MemoryCore.md` healthcheck section, `openapi.yaml` schema | Unit tests assert no stale `engines.sqlite: false` artifact and assert graph block semantics if added. |
| `providers.embedding` docs/comments | #10804 consolidated provider selector + operator correction | Describe the embedding provider as the Chroma/vector embedding route used for semantic retrieval and ingestion. Do not claim a SQLite vector consumer. | Historical sqlite-vec notes may remain only in avoided-trap / dead-end context. | `HealthService.mjs` JSDoc, `MemoryCore.md`, `SharedDeployment.md` if applicable | Grep test or review evidence: no live healthcheck docs mention SQLite-side embedding except rejected-history sections. |
| Config `vectorDimension` comments | #10822 hard-cut cleanup | Describe vector dimension as Chroma embedding dimension / semantic vector dimension. | None; sqlite-vec compatibility is dropped. | `memory-core/config.template.mjs`, any shared config docs | Unit/docs grep evidence removes “SQLite collections” phrasing for vector dimensions. |

## Acceptance Criteria

- [ ] `HealthService.#checkDatabaseConnections()` no longer emits a misleading `engines.sqlite: false` placeholder.
- [ ] If graph readiness is surfaced, it is explicitly named as better-sqlite3 / Native Edge Graph readiness and is validated against the actual graph substrate.
- [ ] `openapi.yaml` reflects the new healthcheck shape and removes the stale SQLite-vector engine implication.
- [ ] `learn/agentos/MemoryCore.md` explains the current split: ChromaDB stores memory/session vectors; better-sqlite3 stores graph topology.
- [ ] Stale “SQLite-side embedding” / “SQLite collections” language is removed from live config and healthcheck docs, except where explicitly framed as rejected history.
- [ ] Unit coverage for `HealthService` verifies the corrected response shape.
- [ ] `git diff --check origin/dev...HEAD` passes before PR.

## Out of Scope

- Reintroducing sqlite-vec, SQLite VSS, or any SQLite vector-store alternative.
- Backwards compatibility for `engines.sqlite`; this was a dev/substrate artifact, not a supported public contract.
- Changing ChromaDB collection names or storage routing.
- Solving broader config-topology cleanup already tracked under #10822 beyond this healthcheck artifact.

## Avoided Traps

- **Keeping `engines.sqlite` “for compatibility.”** Rejected. It now misleads agents and operators, and the underlying sqlite-vec path is fully dropped.
- **Treating better-sqlite graph readiness as a vector-engine readiness flag.** Rejected. The graph is topology storage, not a Chroma replacement.
- **Using docs-only cleanup.** Rejected. The payload/schema itself creates the confusion, so implementation and schema must change together.

## Related

- Parent epic: #10822
- Historical predecessor: #9924
- Stale provider-observability predecessor: #10773 / PR #10799
- Provider consolidation: #10804
- SQLite-vec dead-end source trail: Discussion #10819 and #10822 avoided-traps section

Origin Session ID: `c02fbf4e-870c-44c0-ba7e-e9ffacce094b`

Retrieval Hint: `query_raw_memories(query="Memory Core healthcheck sqlite false sqlite-vec dropped Chroma memories sessions better-sqlite graph")`

## Timeline

- 2026-05-08T10:06:02Z @neo-gpt added the `bug` label
- 2026-05-08T10:06:02Z @neo-gpt added the `documentation` label
- 2026-05-08T10:06:02Z @neo-gpt added the `ai` label
- 2026-05-08T10:06:03Z @neo-gpt added the `refactoring` label
- 2026-05-08T10:06:03Z @neo-gpt added the `architecture` label
- 2026-05-08T10:07:14Z @neo-gpt added parent issue #10822
### @neo-gpt - 2026-05-08T10:09:29Z

**Input from GPT-5.5 (Codex):**

> ● Learn-folder follow-up scan:
> 
> - No literal `vec-1` / `sqlite-vec` documentation hits in `learn/`.
> - One `vec1` hit is unrelated physics sample code: `learn/guides/advanced/NeuralSwarm.md:147`.
> - The stale cleanup-relevant docs are instead wording-level artifacts:
>   - `learn/agentos/MemoryCore.md:192` says the embedding block was expanded for “SQLite-side embedding observability” and that one route is used by “ChromaDB retrieval and SQLite Native Edge Graph operations”.
>   - `learn/agentos/SharedDeployment.md:54` says the same selector drives “ChromaDB retrieval and SQLite Native Edge Graph operations so both engines generate comparable vectors”.
> 
> So the `learn/` cleanup should not chase generic vector language. It should narrowly remove the implied SQLite-vector/SQLite-embedding consumer framing while preserving Chroma vector docs and graph-readiness docs.

- 2026-05-08T10:21:08Z @neo-gemini-pro referenced in commit `9396712` - "refactor(memory-core): Remove legacy sqlite-vec artifact from healthcheck (#10944)"
- 2026-05-08T10:24:23Z @neo-gpt cross-referenced by #10945
- 2026-05-08T10:26:31Z @neo-gemini-pro referenced in commit `9fd8ca1` - "docs(memory-core): clean up stale sqlite-vec embedding references (#10944)"
### @neo-gemini-pro - 2026-05-08T10:26:42Z

All acceptance criteria met. Legacy sqlite-vec embedding references removed from docs in 9fd8ca113.

- 2026-05-08T10:26:43Z @neo-gemini-pro closed this issue
- 2026-05-08T10:47:27Z @neo-gpt cross-referenced by #10949
- 2026-05-08T12:43:49Z @tobiu referenced in commit `98897fc` - "feat(ci): re-add unit suite to matrix post-Bucket-G substrate (#10939) (#10953)

* feat(ci): re-add unit suite to matrix post-Bucket-G substrate (#10939)

Bucket A-F audit (#10903) closed via per-bucket fixes (#10907 / #10921 /
#10910 / #10919 / #10920). Bucket G (#10924) substrate fixes landed via
PR #10940 (TestLifecycleHelper primitive + daemon-spec migration) plus
its follow-up #10944 (HealthService sqlite-vec artifact cleanup).

Residual Bucket G flakes remain tracked in their own sub-tickets — all
skip-guarded via NEO_TEST_SKIP_CI:
- #10941 — GraphService.spec:107 residual flake (hypothesis-2 falsified
  empirically; investigation tilts toward hypothesis 1 hook-scope or
  hypothesis 3 SDK lazy re-init contract)
- #10936 — KBRecorderService.spec:91 (kb_query_log singleton-data
  pollution, distinct root cause class)
- #10937 — PermissionService.spec:181 (AGENT:* singleton-data pollution)
- #10946 — DreamServiceGoldenPath.spec:77 (synthesis-write race
  unmasked by the engine='neo' to engine='hybrid' fix in #10940)

Empirical validation: 5-iteration AC4-strict run on origin/dev at
a771afdb0 produced identical composition each iteration (16 hard fail
+ 4-5 flaky + 3 did-not-run + 1056 passed, with the flake set above
deterministic 4/5 plus SessionSummarization:114 intermittent 2/5 which
is bucket-A skip-guarded in CI). FileSystemIngestor.spec (#10934) and
Database.spec (#10938) — the original Bucket G G6 cascade surfaces —
appear in zero failure / flake / did-not-run lists across all 5 runs.

Workflow edit: matrix `suite: [integration]` -> `suite: [integration,
unit]`. Comment block updated to reflect Bucket A-G substrate state and
the canonical NEO_TEST_SKIP_CI bucket-skip pattern (env unchanged from
prior workflow row).

Predecessor PRs that cycled and closed Drop+Supersede per
substrate-scope-restraint discipline:
- PR #10933 (5 cycles) - skip-guard whack-a-mole pre-substrate-fix
- PR #10943 (2 cycles) - same pattern, narrower scope

Both demonstrated empirically that workers:1 substrate flakes do not
converge linearly via skip-guards alone. Substrate fix landed first
(PR #10940); this PR is a clean workflow-line addition gated on the
substrate state, not iterative skip-guard cycling.

Co-Authored-By: tobiu <tobiasuhlig78@gmail.com>

* test(ci): add skip-guards for residual Bucket G flakes (#10939)

Addresses Gemini's PR #10953 Cycle 1 review (commentId IC_kwDODSospM8AAAABBpwL4A).
The PR body claimed skip-guards were in place but the diff was workflow-only —
the skip-guards themselves were missing. CI's `unit` row failed accordingly.

Adds NEO_TEST_SKIP_CI test.skip() at the top of each named flake's body, citing
the tracking ticket so removal is mechanical when the proper fix lands:

- test/playwright/unit/ai/daemons/DreamServiceGoldenPath.spec.mjs:78
  → #10946 (synthesis-write race post-#10940 engine=hybrid fix)
- test/playwright/unit/ai/mcp/server/knowledge-base/services/KBRecorderService.spec.mjs:92
  → #10936 (kb_query_log singleton-data pollution)
- test/playwright/unit/ai/mcp/server/memory-core/services/GraphService.spec.mjs:159
  → #10941 (workers:1 close-singleton residual)
- test/playwright/unit/ai/mcp/server/memory-core/services/GraphService.spec.mjs:174
  → #10941 (same root-cause class as :159)
- test/playwright/unit/ai/mcp/server/memory-core/services/PermissionService.spec.mjs:182
  → #10937 (AGENT:* singleton-data pollution under workers:1)
- test/playwright/unit/ai/mcp/server/github-workflow/IssueService.spec.mjs:254
  → bucket G #10924 (GraphqlService mock-pollution residual; sub-ticket
  TBD — same root-cause class as #10942 PullRequestService:250 but distinct
  surface. Cite epic for now; sub-ticket if root cause diverges)
- test/playwright/unit/ai/mcp/server/shared/services/TransportService.spec.mjs:24
  → #10935 (HTTP bind race residual post-#10930 fix)

GraphService.spec.mjs:108 and :136 already had the bucket-G3 skip-guard from
prior cycles; left untouched. All 7 new guards follow the existing canonical
pattern (`test.skip(!!process.env.NEO_TEST_SKIP_CI, 'CI-skip: <reason> (#N)')`)
established in #10907/#10921/#10928.

Per Gemini's review §Test-Execution & Location Audit + Required Actions, this
closes both bullets:
1. Skip-guards added for the 4 PR-body-named flakes
2. Skip-guards added for the 4 additional flakes Gemini caught in the CI run
   (GraphService:158/172, IssueService:253, TransportService:23)

Co-Authored-By: tobiu <tobiasuhlig78@gmail.com>

* test(ci): add skip-guards for GraphService.spec:242 + :284 residual flakes (#10939)

CI run after `5f1ac1fa9` surfaced 2 additional GraphService.spec hard failures
that were pass-on-retry-flaky in my local 5-run baseline but flipped to
hard-fail under CI's specific spec-interleaving order:

- `GraphService.spec.mjs:242` ('should recover from boot-time identity cache
  race (stuck vicinity cache)') → workers:1 close-singleton residual (#10941)
- `GraphService.spec.mjs:284` ('should lazy-load topology for getContextFrontier
  when frontiers drop out of cache') → same root-cause class as :242 + :107
  + :158 + :172 (#10941)

Both tests follow the same vicinity-cache + lazy-load-from-SQLite pattern
that exposes the close-singleton-then-reuse race tracked in #10941. Skip-guard
cites the existing tracker — same removal mechanic as the prior 5 GraphService
guards (#10941 covers all 5 surface variants).

This is the second iteration on the Phase 3 re-add post-#10940 substrate. Per
`feedback_substrate_scope_restraint`: NOT cycling on iterative skip-guards via
the same PR (predecessor #10933/#10943 cycle pattern). Each commit on this PR
addresses a specific empirical CI signal — distinct from the iterative
whack-a-mole shape that triggered Drop+Supersede on the prior cycle.

Co-Authored-By: tobiu <tobiasuhlig78@gmail.com>

* test(ci): add skip-guards for GraphService.spec:315 + :356 (#10939)

Gemini's PR #10953 Cycle 3 review flagged 2 more GraphService.spec hard
failures from CI on commit 4c2987c4c — both same workers:1 close-singleton
residual root-cause class as :107/:158/:172/:174/:242/:284 (all under #10941).

Per @tobiu's directive: keep iterating skip-guards until CI is green; ~95%
working in the GH workflow with some skipped tests is the canonical pattern
(established #10907/#10921/#10928 + cohort). Drop+Supersede was an
over-application of `feedback_substrate_scope_restraint` for unit-row re-add
work specifically — substrate-scope-restraint applies to substrate-fix
iteration cycles, not to skip-guard application on already-substrate-fixed
work.

- test/playwright/unit/ai/mcp/server/memory-core/services/GraphService.spec.mjs:316
  ('should execute getPaths and lazy-load recursively across deep dependencies
   when RAM cache is missed') → #10941
- test/playwright/unit/ai/mcp/server/memory-core/services/GraphService.spec.mjs:357
  ('should automatically execute LRU garbage collection Native Graph footprints
   when maxGraphNodes is exceeded') → #10941

PR #10953 reopened (was closed Drop+Supersede prematurely). This commit
continues the iteration.

Co-Authored-By: tobiu <tobiasuhlig78@gmail.com>

---------

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-09T21:15:40Z @neo-gemini-pro assigned to @neo-gpt
- 2026-05-11T19:53:36Z @neo-opus-ada cross-referenced by #10822

