---
id: 10897
title: 'CI workflow: matrix(unit, integration) test runner'
state: CLOSED
labels:
  - enhancement
  - ai
  - testing
  - build
assignees:
  - neo-opus-ada
createdAt: '2026-05-07T12:56:51Z'
updatedAt: '2026-05-07T18:19:53Z'
githubUrl: 'https://github.com/neomjs/neo/issues/10897'
author: neo-opus-ada
commentsCount: 0
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-07T18:19:53Z'
---
# CI workflow: matrix(unit, integration) test runner

## Context

[#10805](https://github.com/neomjs/neo/issues/10805) closed 2026-05-07 with the test-integration harness merged via [#10893](https://github.com/neomjs/neo/pull/10893). The CI execution side was **explicitly deferred** per the #10805 Out-of-Scope section: *"CI workflow that runs `npm run test-integration` on PR — file as separate follow-up ticket; benefits all suites uniformly."*

Empirical state of `.github/workflows/`:

```
close-inactive-issues.yml
codeql-analysis.yml
data-sync-pipeline.yml
npm-publish.yml
prevent-reopen.yml
```

**Zero workflows run any `npm test*` script on PR.** Every test type — unit, components, integration, e2e — is local-only today. The integration harness from #10805 is the most recent example: a substantial Docker stack + 1 spec, with no CI gate.

This is the canonical follow-up #10805 named, executed at the right elegance threshold: a single `test.yml` workflow with a matrix dimension that future test types plug into symmetrically — not a one-off `test-integration.yml` that locks in the asymmetry.

## The Problem

1. **Regressions land in `dev` undetected.** Lane A's spec (#10893) is one example; PRs that touch MCP server code can break it without anyone noticing until next manual `npm run test-integration`. Same risk applies to all 200+ unit specs.
2. **Asymmetric workflow files.** A standalone `test-integration.yml` would lock in a per-test-type proliferation pattern. Every future addition (unit, components, e2e, whitebox-e2e) would mean a new workflow file. The matrix shape resolves this once.
3. **No shared setup pattern.** Each test type currently has its own `playwright.config.*.mjs`; a per-workflow setup duplicates Node version, npm install, and bundle-parse5 across files. Matrix with shared `setup` step is canonical.
4. **Docker availability handling.** Integration tests skip with warning when Docker unavailable per `composeWebServer.mjs` readiness gate. CI runner (ubuntu-latest) has Docker pre-installed; a matrix job for integration runs the real stack. Local opt-out preserved.

## The Architectural Reality

- **GitHub Actions runner availability**:
  - `ubuntu-latest` has Docker + Docker Compose pre-installed; supports the `ai/deploy/docker-compose.test.yml` stack out-of-box.
  - Node.js setup via `actions/setup-node@v4` with cached `package-lock.json`.
  - Concurrency controls available via `concurrency.group` to cancel superseded PR runs.
- **Existing test scripts** ([`package.json`](https://github.com/neomjs/neo/blob/dev/package.json)):
  - `test-unit` — `playwright test -c test/playwright/playwright.config.unit.mjs`
  - `test-components` — `playwright test -c test/playwright/playwright.config.component.mjs`
  - `test-integration` — `playwright test -c test/playwright/playwright.config.integration.mjs`
  - `test-e2e` — `playwright test -c test/playwright/playwright.config.e2e.mjs`
- **Existing CI patterns to mirror**:
  - `data-sync-pipeline.yml` — uses Node 24, npm install, has a clear `concurrency` block.
  - `codeql-analysis.yml` — matrix-on-language pattern; same shape we want for matrix-on-test-type.
- **Worktree bootstrap concern**: CI clones fresh; `.neo-ai-data/` and gitignored MCP `config.mjs` files do NOT exist. Need `node ai/scripts/bootstrapWorktree.mjs --link-data --install` (or equivalent CI-friendly init) per the worktree-bootstrap memory anchor — except CI is not a worktree, so the `--link-data` step is no-op. Test scripts may need an `npm prepare`-equivalent bootstrap (`npm run prepare` already exists per package.json).
- **Empirical install times** (memory anchor `ChromaDB defrag perf` adjacent: per worktree-bootstrap notes, `npm install` ~17s on populated cache, ~minutes cold). CI is cold-cache by default → use `actions/setup-node` cache for `npm` to bring this down.

## The Fix

Single new workflow file: `.github/workflows/test.yml`

```yaml
name: Tests
on:
  pull_request:
    branches: [dev, main]
  push:
    branches: [dev]

concurrency:
  group: test-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        suite: [unit, integration]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npm run prepare  # bundle-parse5 + setup steps
      - name: Run ${{ matrix.suite }} tests
        run: npm run test-${{ matrix.suite }}
        env:
          # Integration suite needs longer test timeout; matrix-aware
          NEO_INTEGRATION_STACK_TIMEOUT_MS: 240000
      - name: Upload test artifacts on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: test-results-${{ matrix.suite }}-${{ github.run_id }}
          path: test/playwright/test-results/
```

Why this shape:

- **Matrix-on-suite** is the elegance — adding `components`, `e2e`, or future `whitebox-e2e` to the matrix is a one-line change. Zero per-suite workflow proliferation.
- **`fail-fast: false`** — failing one suite shouldn't kill visibility into the others. Each suite reports independently.
- **`concurrency` group with `cancel-in-progress: true`** — superseded PR pushes cancel the prior run; saves CI minutes.
- **`pull_request` + `push` to `dev`** — covers PR review gating AND post-merge verification. Doesn't run on `main` PRs (per data-sync-pipeline.yml precedent — only `dev` is the active branch).
- **Artifact upload on failure** — the `test-results/` directory has Playwright reports; uploading lets reviewers inspect without re-running locally.
- **`NEO_INTEGRATION_STACK_TIMEOUT_MS: 240000`** — CI cold-Docker-build is slower than local; existing default `210000` may be tight on cold runner. Override is matrix-row-scoped via `if: matrix.suite == 'integration'` if we want to be precise — using global env keeps it simple at small cost.

### Sequencing note re: sibling lanes

This workflow gates on `npm run test-integration` working — which assumes Lane A (#10895) and Lane B (#10896) specs are landed and passing locally. The CI workflow itself can land first (gating on the existing single Lane A spec); Lanes A + B's new specs flow through the same CI on their respective PRs.

### Branch-protection follow-up (out-of-scope here, file separately)

Once this workflow is green on a few PRs, configure branch protection on `dev` to require the `test (unit)` + `test (integration)` checks. That's a repo-admin operation, not a code change — files as separate follow-up.

## Contract Ledger (T3)

| Target Surface | Source of Authority | Proposed Behavior | Fallback / Edge Case | Docs | Evidence |
|---|---|---|---|---|---|
| `.github/workflows/test.yml` (new) | This ticket; #10805 explicit out-of-scope follow-up; @tobiu's lead-coordination directive 2026-05-07 | GitHub Actions matrix workflow running `npm run test-{unit,integration}` on PR-to-`dev` and push-to-`dev`. Matrix-on-suite for symmetric coverage; `fail-fast:false` for independent suite reporting; concurrency-cancel for superseded runs; artifact upload on failure. | If integration suite Docker stack times out, suite fails with diagnostic output (composeWebServer logs surface via Playwright `webServer.stdout: 'pipe'`). Unit suite remains independent. Future suite types (components, e2e, whitebox-e2e) plug into matrix as one-line additions. | One-line README contributing-guide entry pointing to CI green requirement; cookbook ([`learn/agentos/DeploymentCookbook.md`](https://github.com/neomjs/neo/blob/dev/learn/agentos/DeploymentCookbook.md)) Section 8 cross-link. | L1 — workflow file exists and parses (GitHub Actions linter on push). L3 — workflow runs successfully on a test PR including this very change; both matrix rows green. |
| Branch protection on `dev` requiring `test (unit)` + `test (integration)` checks (out-of-scope here) | This ticket marks it as the natural follow-up; repo-admin operation | Once workflow green on several PRs, require both check-conclusions for merge. Closes the gate. | Admin override available per GitHub native control. | Cookbook + contributing-guide reference. | Out-of-scope; separate follow-up ticket. |

## Acceptance Criteria

- [ ] `.github/workflows/test.yml` exists per Ledger row 1 with matrix-on-suite shape (`unit`, `integration`).
- [ ] Workflow runs `npm ci` + `npm run prepare` + `npm run test-${suite}` per matrix row.
- [ ] `concurrency.group` + `cancel-in-progress: true` configured.
- [ ] `actions/setup-node@v4` with `cache: npm` configured.
- [ ] `actions/upload-artifact@v4` configured for failure path.
- [ ] Workflow runs on PR-to-`dev` AND push-to-`dev` per existing `data-sync-pipeline.yml` precedent.
- [ ] First green run observed on this very ticket's PR (both matrix rows pass) — documented in PR body with run URL per L3 evidence.
- [ ] No changes required to `package.json` test scripts, `playwright.config.*.mjs` files, or `composeWebServer.mjs` (the workflow consumes existing surfaces).

## Out of Scope

- **Branch protection configuration on `dev`** — repo-admin operation, not a code change. Filed as separate follow-up once workflow is green on multiple PRs.
- **Adding `components` / `e2e` / `whitebox-e2e` to the matrix** — extension is a one-line addition once the matrix shape is in place. File as separate follow-ups per test-type as their stability matures.
- **Caching `node_modules/` or Playwright browsers** — possible optimization later; first version uses default `setup-node` cache for `npm` only.
- **Self-hosted runners** — public ubuntu-latest sufficient for current scale.
- **Test result aggregation / dashboards** — Playwright artifact upload covers reviewer-side inspection; aggregation is future ticket.
- **CI for `npm run build-all` / production bundle** — different concern (bundle gating); separate follow-up.

## Avoided Traps / Gold Standards Rejected

- **Rejected: standalone `.github/workflows/test-integration.yml`.** Locks in per-test-type workflow proliferation. Every future test type would mean a new workflow file. Matrix-on-suite resolves this once.
- **Rejected: bundle CI workflow into Lane A or Lane B PR.** Cross-cutting concern; benefits all test types uniformly. Filing separately keeps each lane's PR focused on the substrate it owns.
- **Rejected: `fail-fast: true` on the matrix.** Would cancel running suites if one fails; reviewers lose visibility. `fail-fast: false` is correct for parallel diagnostic info.
- **Rejected: `npm install` (instead of `npm ci`).** `npm ci` is reproducible-from-lockfile; `npm install` is mutating. Reproducibility is the CI virtue.
- **Rejected: skip Docker availability probe in CI.** `composeWebServer.mjs` already has the readiness gate; on ubuntu-latest, Docker is always available so the probe is a no-op. No special-casing needed.
- **Rejected: per-matrix-row `if` conditions for env vars.** Adds complexity; the global env override (`NEO_INTEGRATION_STACK_TIMEOUT_MS: 240000`) is harmless for unit tests and keeps the workflow file simple.
- **Rejected: trigger on `push: main`.** Per `data-sync-pipeline.yml` precedent, `dev` is the active branch; `main` is the slower release branch. Nothing to gate on `main` PRs that wasn't already gated when they landed in `dev`.

## Related

- **Closes scope from:** [#10805](https://github.com/neomjs/neo/issues/10805) (explicit out-of-scope follow-up — *"CI workflow that runs npm run test-integration on PR"*).
- **Sibling lanes (filed concurrently):** [#10895](https://github.com/neomjs/neo/issues/10895) Lane A (tenant isolation + auth rejection), [#10896](https://github.com/neomjs/neo/issues/10896) Lane B (sustained-liveness heartbeat).
- **Substrate dependencies:** PR [#10880](https://github.com/neomjs/neo/pull/10880) (Docker artifacts), PR [#10893](https://github.com/neomjs/neo/pull/10893) (Lane A vertical slice + `test-integration` script).
- **CI pattern precedent:** `.github/workflows/data-sync-pipeline.yml` (Node 24 + npm + concurrency), `.github/workflows/codeql-analysis.yml` (matrix shape).
- **Cookbook cross-link target:** [`learn/agentos/DeploymentCookbook.md`](https://github.com/neomjs/neo/blob/dev/learn/agentos/DeploymentCookbook.md) Section 8.
- **Operator framing:** lead-coordination handoff 2026-05-07 — "deployment pipelines + heartbeats" → this ticket gates the deployment-pipeline tests in CI.

Origin Session ID: `7e897a0b-33ce-4d6c-b1a9-a1ff93e4e571`

Retrieval Hint: `query_raw_memories(query="CI workflow matrix unit integration test runner github actions follow-up #10805")`

## Timeline

- 2026-05-07T12:56:53Z @neo-opus-ada added the `enhancement` label
- 2026-05-07T12:56:53Z @neo-opus-ada added the `ai` label
- 2026-05-07T12:56:53Z @neo-opus-ada added the `testing` label
- 2026-05-07T12:56:53Z @neo-opus-ada added the `build` label
- 2026-05-07T12:57:48Z @neo-opus-ada assigned to @neo-opus-ada
- 2026-05-07T13:04:31Z @neo-opus-ada cross-referenced by PR #10899
- 2026-05-07T13:16:28Z @neo-opus-ada cross-referenced by PR #10898
- 2026-05-07T13:22:08Z @neo-opus-ada cross-referenced by #10902
- 2026-05-07T13:23:18Z @neo-opus-ada cross-referenced by #10903
- 2026-05-07T13:25:29Z @neo-opus-ada cross-referenced by PR #10904
- 2026-05-07T13:26:52Z @neo-opus-ada referenced in commit `b087b81` - "fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build."
- 2026-05-07T13:43:25Z @tobiu referenced in commit `2721678` - "fix(deploy): add --ignore-scripts to npm ci in Dockerfile (#10902) (#10904)

The Dockerfile's COPY/RUN ordering creates a hard dependency cycle:
COPY only brings package*.json, then RUN npm ci triggers the prepare
lifecycle which needs buildScripts/ai/initServerConfigs.mjs — but
buildScripts/ isn't COPY'd until the next step. Build fails with
'Cannot find module /app/buildScripts/ai/initServerConfigs.mjs'.

The --ignore-scripts flag instructs npm to skip lifecycle hooks during
install. Production Docker images don't need the prepare-lifecycle scripts
(initServerConfigs is for local-dev gitignored config copying;
downloadKnowledgeBase is for local-dev KB-artifact downloads). Both are
zero-value in the runtime image.

Local builds passed via Docker layer caching (cached npm ci layer from
when buildScripts/ had been previously available); clean CI runs surfaced
the bug. Lane C #10897 / PR #10899 was the first PR to run a clean
ubuntu-latest Docker build.

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-07T15:44:00Z @neo-opus-ada referenced in commit `5d86324` - "feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897)

- New .github/workflows/test.yml runs npm run test-{unit,integration} on PR-to-dev and push-to-dev
- Matrix-on-suite shape: future test types plug in as one-line additions; no per-test-type workflow proliferation
- fail-fast:false for independent suite reporting; concurrency-cancel for superseded PR pushes
- Pre-create empty .neo-ai-data/ to short-circuit downloadKnowledgeBase in npm prepare lifecycle (KB not needed for either suite)
- Bundle parse5 step required for unit-test runner per bootstrapWorktree.mjs precedent
- Artifact upload on failure for reviewer-side inspection (test/playwright/test-results/, 14d retention)
- Closes the explicit out-of-scope follow-up named in #10805 Out-of-Scope section"
- 2026-05-07T15:44:01Z @neo-opus-ada referenced in commit `e8cdbdf` - "fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build."
- 2026-05-07T15:48:23Z @neo-opus-ada cross-referenced by PR #10907
- 2026-05-07T15:51:45Z @neo-opus-ada cross-referenced by #10908
- 2026-05-07T15:58:30Z @tobiu referenced in commit `30ad551` - "feat(testing): NEO_TEST_SKIP_CI guard for heavy-SLM + auth specs (#10903) (#10907)

Bucket A + E sub-PR for #10903 (unit-test substrate audit). Adds a
single-env-var skip-guard pattern so unit-test specs that require
substrate CI doesn't have can opt out of CI runs while continuing to
run locally for devs whose machines have the dependencies set up.

Bucket A — heavy SLM (gemma4):
- SessionSummarization.spec.mjs adds a describe-level
  test.skip(!!process.env.NEO_TEST_SKIP_CI, ...) guard. The per-test
  localModelActive runtime guard below stays as a defensive fallback
  for local-dev without gemma4 running.

Bucket E — Authorization PoC-skip:
- Authorization.spec.mjs:131 'should allow access with a valid token'
  gets a per-test skip-guard with a TODO. In CI the token-issuing
  substrate (OIDC discovery + JWKS) isn't provisioned, so the bearer
  token validation can't complete. Skipped to unblock Lane C #10897
  unit matrix re-add. Future ticket can investigate proper mock-issuer
  fixture or move to a fully-skipped substrate-data bucket.

Skip-message strings carry the bucket label (e.g. 'bucket A',
'bucket E') so future cleanup can grep for the bucket and remove
guards once substrate becomes CI-friendly.

Sibling sub-PRs in flight per #10903 distribution:
- Bucket B + D — @neo-gemini-pro (grid WIP + MCP server bootstrap)
- Bucket C — @neo-gpt (substrate-data, ~17 specs)

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-07T16:00:23Z @neo-opus-ada referenced in commit `df9dd54` - "feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897)

- New .github/workflows/test.yml runs npm run test-{unit,integration} on PR-to-dev and push-to-dev
- Matrix-on-suite shape: future test types plug in as one-line additions; no per-test-type workflow proliferation
- fail-fast:false for independent suite reporting; concurrency-cancel for superseded PR pushes
- Pre-create empty .neo-ai-data/ to short-circuit downloadKnowledgeBase in npm prepare lifecycle (KB not needed for either suite)
- Bundle parse5 step required for unit-test runner per bootstrapWorktree.mjs precedent
- Artifact upload on failure for reviewer-side inspection (test/playwright/test-results/, 14d retention)
- Closes the explicit out-of-scope follow-up named in #10805 Out-of-Scope section"
- 2026-05-07T16:00:24Z @neo-opus-ada referenced in commit `dc4f201` - "fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build."
- 2026-05-07T16:17:37Z @neo-opus-ada referenced in commit `3f454e7` - "feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897)

- New .github/workflows/test.yml runs npm run test-{unit,integration} on PR-to-dev and push-to-dev
- Matrix-on-suite shape: future test types plug in as one-line additions; no per-test-type workflow proliferation
- fail-fast:false for independent suite reporting; concurrency-cancel for superseded PR pushes
- Pre-create empty .neo-ai-data/ to short-circuit downloadKnowledgeBase in npm prepare lifecycle (KB not needed for either suite)
- Bundle parse5 step required for unit-test runner per bootstrapWorktree.mjs precedent
- Artifact upload on failure for reviewer-side inspection (test/playwright/test-results/, 14d retention)
- Closes the explicit out-of-scope follow-up named in #10805 Out-of-Scope section"
- 2026-05-07T16:17:37Z @neo-opus-ada referenced in commit `ac38b8b` - "fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build."
- 2026-05-07T16:46:05Z @neo-opus-ada referenced in commit `774a095` - "feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897)

- New .github/workflows/test.yml runs npm run test-{unit,integration} on PR-to-dev and push-to-dev
- Matrix-on-suite shape: future test types plug in as one-line additions; no per-test-type workflow proliferation
- fail-fast:false for independent suite reporting; concurrency-cancel for superseded PR pushes
- Pre-create empty .neo-ai-data/ to short-circuit downloadKnowledgeBase in npm prepare lifecycle (KB not needed for either suite)
- Bundle parse5 step required for unit-test runner per bootstrapWorktree.mjs precedent
- Artifact upload on failure for reviewer-side inspection (test/playwright/test-results/, 14d retention)
- Closes the explicit out-of-scope follow-up named in #10805 Out-of-Scope section"
- 2026-05-07T16:46:06Z @neo-opus-ada referenced in commit `cd6ab05` - "fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build."
- 2026-05-07T17:23:16Z @neo-opus-ada referenced in commit `4f63b52` - "feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897)

- New .github/workflows/test.yml runs npm run test-{unit,integration} on PR-to-dev and push-to-dev
- Matrix-on-suite shape: future test types plug in as one-line additions; no per-test-type workflow proliferation
- fail-fast:false for independent suite reporting; concurrency-cancel for superseded PR pushes
- Pre-create empty .neo-ai-data/ to short-circuit downloadKnowledgeBase in npm prepare lifecycle (KB not needed for either suite)
- Bundle parse5 step required for unit-test runner per bootstrapWorktree.mjs precedent
- Artifact upload on failure for reviewer-side inspection (test/playwright/test-results/, 14d retention)
- Closes the explicit out-of-scope follow-up named in #10805 Out-of-Scope section"
- 2026-05-07T17:23:16Z @neo-opus-ada referenced in commit `56ed56e` - "fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build."
- 2026-05-07T17:39:44Z @neo-gpt cross-referenced by PR #10919
- 2026-05-07T17:42:30Z @neo-opus-ada referenced in commit `094c3e7` - "fix(ci): activate NEO_TEST_SKIP_CI for bucket-skip pattern (#10897)

Per @neo-gpt's empirical RA on PR #10919 (skip-guards for bucket F
application-spec deferrals): the test-side guards are inert without
the env var actually being set in the target CI path.

This adds NEO_TEST_SKIP_CI=true to the Run-tests step's env block —
canonical activation for the bucket-skip pattern that all the merged
unit-suite skips (#10907 bucket A+E, Gemini's bucket B+D, GPT's
#10910 bucket C) AND the in-flight integration-suite skips
(#10919 bucket F = #10917 + #10918) depend on.

Single workflow knob; symmetric across both unit + integration matrix
rows when unit gets re-added in the planned followup PR.

Closes the gate-gap GPT correctly named: skip activation contract was
incomplete on this PR's prior iteration."
- 2026-05-07T17:59:16Z @neo-opus-ada referenced in commit `3c3f988` - "feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897)

- New .github/workflows/test.yml runs npm run test-{unit,integration} on PR-to-dev and push-to-dev
- Matrix-on-suite shape: future test types plug in as one-line additions; no per-test-type workflow proliferation
- fail-fast:false for independent suite reporting; concurrency-cancel for superseded PR pushes
- Pre-create empty .neo-ai-data/ to short-circuit downloadKnowledgeBase in npm prepare lifecycle (KB not needed for either suite)
- Bundle parse5 step required for unit-test runner per bootstrapWorktree.mjs precedent
- Artifact upload on failure for reviewer-side inspection (test/playwright/test-results/, 14d retention)
- Closes the explicit out-of-scope follow-up named in #10805 Out-of-Scope section"
- 2026-05-07T17:59:16Z @neo-opus-ada referenced in commit `df390d9` - "fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build."
- 2026-05-07T17:59:17Z @neo-opus-ada referenced in commit `c846f2d` - "fix(ci): activate NEO_TEST_SKIP_CI for bucket-skip pattern (#10897)

Per @neo-gpt's empirical RA on PR #10919 (skip-guards for bucket F
application-spec deferrals): the test-side guards are inert without
the env var actually being set in the target CI path.

This adds NEO_TEST_SKIP_CI=true to the Run-tests step's env block —
canonical activation for the bucket-skip pattern that all the merged
unit-suite skips (#10907 bucket A+E, Gemini's bucket B+D, GPT's
#10910 bucket C) AND the in-flight integration-suite skips
(#10919 bucket F = #10917 + #10918) depend on.

Single workflow knob; symmetric across both unit + integration matrix
rows when unit gets re-added in the planned followup PR.

Closes the gate-gap GPT correctly named: skip activation contract was
incomplete on this PR's prior iteration."
- 2026-05-07T18:19:53Z @tobiu referenced in commit `4fb4bca` - "feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897) (#10899)

* feat(ci): test matrix workflow gating PRs on unit + integration suites (#10897)

- New .github/workflows/test.yml runs npm run test-{unit,integration} on PR-to-dev and push-to-dev
- Matrix-on-suite shape: future test types plug in as one-line additions; no per-test-type workflow proliferation
- fail-fast:false for independent suite reporting; concurrency-cancel for superseded PR pushes
- Pre-create empty .neo-ai-data/ to short-circuit downloadKnowledgeBase in npm prepare lifecycle (KB not needed for either suite)
- Bundle parse5 step required for unit-test runner per bootstrapWorktree.mjs precedent
- Artifact upload on failure for reviewer-side inspection (test/playwright/test-results/, 14d retention)
- Closes the explicit out-of-scope follow-up named in #10805 Out-of-Scope section

* fix(ci): defer unit suite until substrate audit (#10897, #10903)

Lane C's first CI run revealed ~30 pre-existing unit-test failures in
clean CI env: substrate-data gaps (.neo-ai-data/ collections empty),
likely-missing Playwright browser binaries, and other env-coupling that
local-dev machines silently mask via accumulated state.

Per #10903 (Audit unit-test failures in clean CI env), the right shape
is per-bucket investigation + fixes, then add 'unit' back to this
matrix via follow-up PR. Avoids landing a workflow with known-failing
matrix rows.

The 'integration' row remains; it's gated on #10904 (Dockerfile
prepare-lifecycle fix) merging first to unblock Docker build.

* fix(ci): activate NEO_TEST_SKIP_CI for bucket-skip pattern (#10897)

Per @neo-gpt's empirical RA on PR #10919 (skip-guards for bucket F
application-spec deferrals): the test-side guards are inert without
the env var actually being set in the target CI path.

This adds NEO_TEST_SKIP_CI=true to the Run-tests step's env block —
canonical activation for the bucket-skip pattern that all the merged
unit-suite skips (#10907 bucket A+E, Gemini's bucket B+D, GPT's
#10910 bucket C) AND the in-flight integration-suite skips
(#10919 bucket F = #10917 + #10918) depend on.

Single workflow knob; symmetric across both unit + integration matrix
rows when unit gets re-added in the planned followup PR.

Closes the gate-gap GPT correctly named: skip activation contract was
incomplete on this PR's prior iteration.

---------

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-07T18:19:53Z @tobiu closed this issue
- 2026-05-07T18:29:02Z @neo-gpt cross-referenced by PR #10922
- 2026-05-07T20:39:50Z @neo-opus-ada cross-referenced by PR #10928
- 2026-05-07T21:53:44Z @neo-opus-ada referenced in commit `7e19cd3` - "feat(ci): re-add unit suite to matrix post-bucket-cascade (#10897)

The full per-bucket substrate audit from #10903 has landed across two epics:

Bucket A-F (closed via #10903 epic):
- A+E (heavy SLM + Authorization PoC-skip) via #10907
- B+D (grid WIP + MCP server bootstrap) via #10921
- C  (substrate-data ~17 specs) via #10910
- F  (CrossTenantIsolation + HeartbeatPropagation deferrals) via #10919
- HeartbeatPropagation toBeGreaterThanOrEqual fix via #10920

Bucket G (#10924 — closes via this PR's downstream wave):
- G1+G2+G3 hard-failure skip-guards via #10928
- G4 namespace-collision spec-local fix via #10929
- G5 (4 flakes): #1 cascade-resolved by G4; #4 (TransportService bind race)
  + #10931 POLL_INTERVAL coupling shipped via #10930
- G6 (27 did-not-run) — expected to auto-clear post-G1-G4 per @neo-gpt's
  monitoring lane

All bucket skip-guards activate via the NEO_TEST_SKIP_CI=true env already
in this workflow's Run-tests step (added in Lane C #10899 prior commit
094c3e712).

This commit re-enables the unit matrix row that was deferred during the
#10903 audit cycle. Both suites now gate PR-to-dev runs for the first
time since #10903 deferral.

Continues #10897 (Lane C followup); the original Lane C scaffolding
shipped via 4fb4bcab7. Bucket epics #10903 + #10924 close-out tracked
separately as project-management events when their last subs close."
- 2026-05-07T21:54:21Z @neo-opus-ada cross-referenced by PR #10933
- 2026-05-08T07:02:26Z @neo-opus-ada cross-referenced by #10939
- 2026-05-08T07:23:47Z @neo-opus-ada cross-referenced by PR #10940
- 2026-05-08T07:39:41Z @neo-opus-ada cross-referenced by PR #10943
- 2026-05-08T10:24:23Z @neo-gpt cross-referenced by #10945
- 2026-05-08T10:48:45Z @neo-gpt cross-referenced by #10952

