---
id: 11163
title: 'bootstrapWorktree: generate dist/parse5.mjs (+ minimal build artifacts) so worktree can run full unit suite'
state: CLOSED
labels:
  - enhancement
  - ai
  - build
  - model-experience
assignees:
  - neo-gemini-pro
createdAt: '2026-05-10T23:37:45Z'
updatedAt: '2026-05-11T00:33:08Z'
githubUrl: 'https://github.com/neomjs/neo/issues/11163'
author: neo-opus-ada
commentsCount: 1
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-11T00:33:08Z'
---
# bootstrapWorktree: generate dist/parse5.mjs (+ minimal build artifacts) so worktree can run full unit suite

## Context

`ai/scripts/bootstrapWorktree.mjs` currently scopes to: gitignored `ai/mcp/server/<name>/config.mjs` copies + gitignored `.neo-ai-data/` data subdirs symlinks. It does NOT generate build artifacts in the worktree's `dist/` directory.

**Empirical anchor 2026-05-10:** running `CI=true npm run test-unit` (full suite) in a Claude Code worktree fails with:
```
Error: Cannot find module '/path/to/.claude/worktrees/<name>/dist/parse5.mjs'
  imported from /path/to/.claude/worktrees/<name>/src/functional/util/HtmlTemplateProcessor.mjs
```

Operator-flagged hypothesis: *"dist/parse5 probably get generated by build all. might be a `ai/scripts/bootstrapWorktree.mjs` item. mostly important for the left hemisphere, but could always be done (takes less than 30s on this machine)."*

Verified:
- `dist/parse5.mjs` generated by `npm run bundle-parse5` per `buildScripts/README.md`
- `buildScripts/util/templateBuildProcessor.mjs:2` imports `from '../../dist/parse5.mjs'`
- Main checkout `/Users/Shared/github/neomjs/neo/dist/` has the full set: parse5, esm/, development/, production/, highlight/
- Fresh worktree `dist/` has only `ai-knowledge-base.jsonl` (bootstrapped per current scope)
- Workaround today: scope test runs to AI-only substrate (`npm run test-unit -- test/playwright/unit/ai/`) which avoids parse5-dependent specs

## The Problem

Without dist artifacts in a fresh worktree:
1. Full unit-suite verification (`CI=true npm run test-unit`) fails immediately
2. Any spec importing `HtmlTemplateProcessor` (functional component substrate) fails
3. Workaround: narrower spec-pattern scoping — but this leaves the worktree without true CI-equivalent verification capability
4. Recurring friction across Claude Code worktree sessions; operator-flagged today after running into it for the second time this session

## The Architectural Reality

`bootstrapWorktree.mjs` distinguishes (per its own JSDoc):
- **Source code:** must be REAL COPIES (symlinks cause `Neo.setupClass` namespace collisions when worktree-local module + main-checkout module both register)
- **Data directories:** SAFE TO SYMLINK (no ESM import chains; better-sqlite3 + Chroma open by path transparently)

`dist/parse5.mjs` is a special case — it's a BUNDLED THIRD-PARTY module (not a Neo class), but it IS ESM-imported by `templateBuildProcessor.mjs`. Per the rule's spirit (namespace-collision), parse5 itself doesn't trigger collisions (it doesn't extend Neo.core.Base or register namespaces), so symlinking IS technically safe. But **generating dist artifacts in the worktree** is cleaner — keeps worktree self-contained, removes semantic-edge-case risk.

Same applies to other dist artifacts (esm/, development/, production/, highlight/) — bundled outputs from various `npm run build-*` scripts, ESM-imported by tests + runtime.

## The Fix

Extend `ai/scripts/bootstrapWorktree.mjs` with a dist-generation step:

```js
// After config copy + data symlinks + gitignored-file symlinks:
console.log('[bootstrap] Generating dist artifacts...');
spawnSync('npm', ['run', 'bundle-parse5'], {cwd: workTreeRoot, stdio: 'inherit'});
// Optionally: trigger broader build-all if operator wants full dist parity with main checkout
```

**Scope decision points:**
1. **Minimal scope:** just `bundle-parse5` (sufficient for unit-suite-runs that import HtmlTemplateProcessor)
2. **Full scope:** entire `build-all` run (covers all dist artifacts; takes ~few minutes; matches main-checkout parity exactly)
3. **Opt-in via CLI flag:** `--build-dist` flag, off by default (minimizes default bootstrap time)

Operator-framing favors option 1 (<30s) or option 3 (opt-in for full).

## Acceptance Criteria

- [ ] **AC1:** `bootstrapWorktree.mjs` generates `dist/parse5.mjs` (at minimum) post-config-copy + post-data-symlinks
- [ ] **AC2:** Fresh worktree post-bootstrap can run `CI=true npm run test-unit` without "Cannot find module dist/parse5.mjs" error (verify empirically)
- [ ] **AC3:** Existing bootstrap behavior unchanged (config copy + data symlinks + gitignored-file symlinks all preserved)
- [ ] **AC4 (scope decision-AC):** Either default-on minimal-scope (just bundle-parse5) OR opt-in CLI flag for broader build-all — implementation-time decision
- [ ] **AC5:** JSDoc updated to document the new dist-generation step + scope decision rationale

## Out of Scope

- **Symlinking dist/ to main checkout** — rejected; mixes data semantics with build artifacts; potential edge cases with namespace collisions for any ESM-imported dist module
- **Full build-all by default** — too aggressive; minimal scope is sufficient for unblocking unit-suite + keeps default bootstrap time bounded
- **Generating per-app build outputs** — too aggressive; only platform-bundled-modules (parse5) needed for test substrate

## Avoided Traps

- **Skip-as-low-priority** — operator-flagged today after empirical re-occurrence; small scope; high recurring-friction-reduction ROI for every Claude Code worktree session
- **Bundle into broader bootstrap refactor** — minimal change; bounded scope; no need to expand
- **Symlink instead of generate** — symlink semantics for ESM-imported modules has namespace-collision risk per bootstrap's own JSDoc rule, even though parse5 specifically wouldn't trigger it. Generation is cleaner architecturally.

## Related

- **Operator hypothesis surface:** 2026-05-10 session, *"dist/parse5 probably get generated by build all. might be a ai/scripts/bootstrapWorktree.mjs item."*
- **Empirical recurrence:** 2 cases this session of full-unit-suite invocations failing on dist/parse5.mjs missing
- **bootstrap JSDoc source-vs-data symlink rule:** `ai/scripts/bootstrapWorktree.mjs` head comment
- **parse5 build script:** `buildScripts/README.md` + `npm run bundle-parse5`

Origin Session ID: c2912891-b459-4a03-b2af-154d5e264df1

Retrieval Hint: `query_raw_memories(query="bootstrapWorktree dist parse5 generation unit suite full-run worktree friction")`

## Timeline

- 2026-05-10T23:37:46Z @neo-opus-ada added the `enhancement` label
- 2026-05-10T23:37:46Z @neo-opus-ada added the `ai` label
- 2026-05-10T23:37:46Z @neo-opus-ada added the `build` label
- 2026-05-10T23:37:47Z @neo-opus-ada added the `model-experience` label
### @neo-opus-ada - 2026-05-10T23:43:20Z

## V-B-A 2026-05-10: operator preferred scope = full `npm run build-all`

Operator directive: *"cleanest fix is npm run build-all => then you have everything. especially left hemisphere. vba please."*

**V-B-A via code-reading** confirms hypothesis exactly. `npm run build-all` resolves to `node ./buildScripts/build/all.mjs -f -n`, which orchestrates:

| Step | Generates |
|---|---|
| `npm i` | deps |
| `parse5.mjs` build | `dist/parse5.mjs` ← the specific file the original empirical anchor hit |
| `highlightJs.mjs` build | `dist/highlight/` |
| `themes.mjs` build | theme assets |
| `buildThreads.mjs` | thread bundles |
| `esmodules.mjs` build | `dist/esm/` |
| `generate-docs-json` + copyFolder | `dist/development/docs/` + `dist/production/docs/` |
| seo copy | seo assets |

(verified: `buildScripts/build/all.mjs:131-182` shows the full spawnSync chain)

**AC4 scope decision** (was "minimal-scope vs opt-in-full" in original body) now **resolved to full-scope `npm run build-all`**: operator-preferred, single-command, broader coverage, <30s on operator's machine.

**Updated AC1:** `bootstrapWorktree.mjs` runs `npm run build-all` post-config-copy + post-data-symlinks. Generates ALL dist artifacts (parse5 + esm + highlight + themes + thread bundles + docs + seo), not just parse5.

**Trade-off acknowledged:** broader scope means longer bootstrap time. Operator empirically confirms <30s on his M-series machine. If bootstrap time becomes a friction point on slower hardware, optional `--minimal-dist` flag could land as scope-extension (just bundle-parse5 for the test-suite-unblock case). Defer until empirical friction surfaces.

**Out-of-scope adjustment:** removing the prior "Full build-all by default — too aggressive" Avoided-Trap entry; operator-preference reverses that judgment.

Updating AC1 / AC4 / Out-of-Scope sections in the ticket body to reflect this. Anyone picking up #11163 should implement `npm run build-all` invocation in bootstrap, not the minimal-scope I originally drafted.

— @neo-opus-ada (Origin Session: c2912891-b459-4a03-b2af-154d5e264df1)


- 2026-05-10T23:46:36Z @neo-gemini-pro assigned to @neo-gemini-pro
- 2026-05-11T00:16:25Z @neo-gemini-pro cross-referenced by PR #11170
- 2026-05-11T00:33:08Z @tobiu referenced in commit `04d9415` - "docs(worktree): document #11163 build-all scope decision in bootstrapWorktree (#11163) (#11170)

Co-authored-by: neo-gemini-pro <neo-gemini-pro@users.noreply.github.com>"
- 2026-05-11T00:33:08Z @tobiu closed this issue
- 2026-05-11T00:41:26Z @neo-opus-ada cross-referenced by PR #11167

