---
id: 11150
title: 'Harden production-scale ai:restore path: Neo bootstrap + chunked Chroma upsert + streaming validateBundle'
state: CLOSED
labels:
  - bug
  - ai
  - architecture
  - model-experience
assignees:
  - neo-opus-ada
createdAt: '2026-05-10T19:57:36Z'
updatedAt: '2026-06-07T17:10:13Z'
githubUrl: 'https://github.com/neomjs/neo/issues/11150'
author: neo-opus-ada
commentsCount: 0
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-10T20:36:23Z'
---
# Harden production-scale ai:restore path: Neo bootstrap + chunked Chroma upsert + streaming validateBundle

## Premise

@neo-gpt's coordination after the 2026-05-10 graph + Chroma restoration (commentId via DM): three production-scale bugs were exposed when restoring a real-world backup (Apr 28 mc bundle ~10k records, May 10 01:11 kb bundle 586MB). All three blocked the restore until patched live; should land on `dev` as a follow-up so the `ai:restore` primitive holds at scale.

## Empirical anchors (2026-05-10 restoration)

1. **`buildScripts/ai/restore.mjs` Neo bootstrap missing**: invoking `node ./buildScripts/ai/restore.mjs ...` failed at `src/core/Compare.mjs:166` with `ReferenceError: Neo is not defined`. Cause: importing `ai/services.mjs` chains to core classes that call `Neo.gatekeep(...)`, but `restore.mjs` does NOT import `src/Neo.mjs` + `src/core/_export.mjs` first. The unit-test pattern (`restore-filters.spec.mjs`, `IssueService.spec.mjs`) already documents the bootstrap requirement; build-script entrypoint missed it.

2. **`Memory_DatabaseService.#importMemories` Chroma upsert batch limit**: importing 9,244-record Apr 28 mc backup hit `Record set length 9244 exceeds max batch size 5461`. After chunking at 4000, hit `413: Payload Too Large` from the HTTP layer. Empirically successful chunk size: **250 records** (with 4096-dim qwen3 embeddings ≈ 35-40KB per record → ~10MB body, well within both record-count cap and HTTP body size cap).

3. **`buildScripts/ai/restore.mjs validateBundle()` V8 max-string-length**: validating the May 10 01:11 backup hit `Cannot create a string longer than 0x1fffffe8 characters` (512MB cap). Cause: `validateBundle` reads each JSONL file with `fs.readFile(path, 'utf8')` then `.split('\\n')` to grab the first non-empty line for parseability sanity-check. The 586MB kb backup file overflows V8's max string length.

## Prescription

Three minimal fixes scoped to `buildScripts/ai/restore.mjs` + `ai/services/memory-core/DatabaseService.mjs`:

### Fix 1 — Neo bootstrap in restore.mjs

```js
// Bootstrap Neo namespace BEFORE importing services that depend on Neo.gatekeep
import Neo              from '../../src/Neo.mjs';
import * as core        from '../../src/core/_export.mjs';
// existing imports …
import { … } from '../../ai/services.mjs';
```

### Fix 2 — Chunked Chroma upsert in #importMemories

Replace single `collection.upsert({ ids: records.map(...) })` with chunked loop at named constant `CHROMA_UPSERT_CHUNK_SIZE = 250`. Per-chunk progress logged. Counters preserved (truthful per #11141 contract).

### Fix 3 — Streaming validateBundle first-line read

Replace `fs.readFile(path, 'utf8').split('\\n').find(...)` with a `readline.createInterface` first-line pass. O(1) memory regardless of file size. Same parseability sanity check, no V8 string-cap collision.

## Acceptance Criteria

- [ ] `restore.mjs` imports `src/Neo.mjs` + `src/core/_export.mjs` before any service import
- [ ] `DatabaseService.#importMemories` chunks Chroma `collection.upsert()` at named `CHROMA_UPSERT_CHUNK_SIZE = 250` constant; truthful counters preserved
- [ ] `validateBundle()` first-line JSONL parseability check uses streaming readline (no full-file readFile)
- [ ] Unit test: `Memory_DatabaseService.#importMemories` chunked-upsert behavior (mocked Chroma collection: assert N/250 batched calls for N-record JSONL)
- [ ] Unit test: `validateBundle()` smoke test with synthetic JSONL (without giant fixture; verify streaming path doesn't read whole file)
- [ ] Unit test: `restore.mjs` bootstrap loads without `Neo.gatekeep` error in fresh node process
- [ ] Empirical: `npm run ai:restore` against a >500MB JSONL bundle completes without ERR_STRING_TOO_LONG

## Avoided Traps

| Considered | Rejected | Rationale |
|---|---|---|
| Bundle in #11141 / #11144 | Reject | Different bug surface (operational scale, not semantic); cleaner separate PR. |
| Bypass `ai/services.mjs` to skip Neo bootstrap | Reject | SDK boundary discipline (#10845) — services.mjs IS the canonical entrypoint. Add bootstrap, don't bypass. |
| Auto-detect optimal chunk size | Reject for this PR | Static `250` is empirically successful; adaptive chunking (probe Chroma config) is YAGNI for today's failure mode. |
| Skip validateBundle parseability for files >512MB | Reject | Lose sanity-check signal entirely. Streaming first-line is cleaner + universally applicable. |

## Empirical Anchors

- 2026-05-10 graph + Chroma restoration: full recovery scope 1,068→10,312 memories / 65→876 summaries / 3,545→23,514 graph nodes
- @neo-gpt's coordination DM: confirmed all 3 fixes absent on origin/dev post-#11142/#11143 merges
- `feedback_npx_bypass_test_isolation` analog at substrate-restore layer: scale-related bugs hide until production-scale data exercises them

## Cross-Family Review

@neo-gpt for cross-family review (subsystem-familiarity: he flagged the gaps + suggested the PR shape; same review-coherence as #11143/#11146).

— @neo-opus-ada

## Timeline

- 2026-05-10T19:57:38Z @neo-opus-ada added the `bug` label
- 2026-05-10T19:57:38Z @neo-opus-ada added the `ai` label
- 2026-05-10T19:57:38Z @neo-opus-ada added the `architecture` label
- 2026-05-10T19:57:38Z @neo-opus-ada added the `model-experience` label
- 2026-05-10T19:57:42Z @neo-opus-ada assigned to @neo-opus-ada
- 2026-05-10T19:59:03Z @neo-opus-ada cross-referenced by PR #11151
- 2026-05-10T20:08:52Z @neo-opus-ada referenced in commit `7f4947e` - "test(ai-restore): cycle-1 RA — 3 regression tests for #11150 hardening

Per @neo-gpt's PR #11151 cycle-1 review (id 4259848893): implementation accepted;
blocker is test coverage for the 3 hardening fixes. Adds
`test/playwright/unit/ai/buildScripts/restore-hardening.spec.mjs` with one test
per fix:

**Test 1 — Bootstrap regression (Neo namespace):**
Spawn `node ./buildScripts/ai/restore.mjs <missing-bundle>` in fresh process.
Assert stderr contains validateBundle's "Bundle directory not found" error
(proves Neo bootstrap completed) AND does NOT contain "Neo is not defined" or
"Neo.gatekeep is not a function" (the bug surface).

**Test 2 — Streaming validateBundle:**
Synthetic 5-subdir bundle with one tiny *.jsonl per subdir. Spy on
`fs-extra.readFile`; call validateBundle; assert NO JSONL files appear in the
readFile call list (the bug path overflows V8 string cap on full-file reads).
Streaming readline path produces zero readFile calls for JSONL.

**Test 3 — Chunked Chroma upsert in #importMemories:**
Mock `Memory_StorageRouter.getMemoryCollection` to return a fake with
upsert-call-recording. Synthetic 1000-record JSONL bundle. Invoke
`Memory_DatabaseService.manageDatabaseBackup({action: 'import', mode: 'merge'})`.
Assert exactly ⌈1000/250⌉ = 4 upsert calls; sum = 1000; each call size ≤ 250."
- 2026-05-10T20:36:23Z @tobiu referenced in commit `0596953` - "fix(ai-restore): harden production-scale restore path after May 10 recovery (#11150) (#11151)

* fix(ai-restore): harden production-scale restore path after May 10 recovery (#11150)

Three production-scale bugs surfaced when restoring real-world backups during
the 2026-05-10 graph + Chroma recovery:

**Fix 1 — Neo bootstrap missing in restore.mjs:**
`node ./buildScripts/ai/restore.mjs ...` failed at `Compare.mjs:166` with
`ReferenceError: Neo is not defined`. Cause: `ai/services.mjs` chains to core
classes calling `Neo.gatekeep(...)`, but `restore.mjs` did not import
`src/Neo.mjs` + `src/core/_export.mjs` first. Build-script entrypoint missed
the bootstrap that AI unit-tests already document (e.g.,
`restore-filters.spec.mjs`, `IssueService.spec.mjs`). Added the imports.

**Fix 2 — Chroma upsert batch limit in #importMemories:**
Importing 9,244-record Apr 28 mc backup hit
`Record set length 9244 exceeds max batch size 5461`. Chunking at 4000 then
hit `413: Payload Too Large` from the HTTP layer. Empirically successful
chunk size: 250 records (with 4096-dim qwen3 embeddings ~32KB each + document
text → ~10MB body, well within both record-count cap and HTTP body size cap).
Added named constant `CHROMA_UPSERT_CHUNK_SIZE = 250` + per-chunk progress log.
Truthful counters preserved (per #11141 contract).

**Fix 3 — V8 max-string-length in validateBundle():**
Validating the May 10 01:11 backup hit
`Cannot create a string longer than 0x1fffffe8 characters` (512MB cap). Cause:
`validateBundle` read each JSONL file whole with `fs.readFile(path, 'utf8')`
then `.split('\"
- 2026-05-10T20:36:23Z @tobiu closed this issue

