# HA-HR1 H15 Hybrid Live Evidence

Date: 2026-05-08

This note records the first live H15 hybrid-router measurements for
`qwen3-coder:30b` plus Codex frontier repair.

## Decision

H15 endpoint work should use `frontier-only` as the current full-task baseline,
not local-only or local-first hybrid.

Reason:

- the first run exposed a harness command-wrapping bug, not a model result;
- after the wrapper fix, the local lane reached `qwen3-coder:30b`;
- the local lane then failed from Ollama runtime resource limits, not task logic;
- frontier repair completed the implementation and the private H15 oracle passed.

The successful repair is positive evidence for the hybrid route shape, but the
later frontier-only baseline is stronger for H15 on this machine because it used
one frontier call instead of three frontier calls plus a failed local call.

## `HA-HR1-H15-hybrid-qwen-coder-001-20260508T110000Z`

This run used the H15 hybrid-router profile with:

- local worker: `ollama`, `qwen3-coder:30b`, PowerShell stdin transport;
- frontier classify, repair, and review: Codex;
- local resource sampling: `powershell.exe -NoProfile -Command "ollama ps"`;
- private oracle: `h15-api-endpoint-oracle.mjs`.

Outcome:

- private oracle: passed after frontier repair;
- local-bulk exit: `1`;
- local-bulk wall time: `0.025s`;
- local resource samples: one empty sample, no model loaded;
- local failure: `Cannot find module '/bin/cli.mjs'`.

Decision: classify this run as harness evidence only. The sampled live-step wrapper
expanded `$repo` before the nested `bash -lc` command could see its own assignment,
so the local lane never reached Ollama. Commit `e5b9184` fixes this by executing
sampled live commands as argv arrays instead of parsing display strings through
another shell.

## `HA-HR1-H15-hybrid-qwen-coder-002-20260508T111449Z`

This rerun used commit `e5b9184` and the same H15 hybrid command shape.

Step summary:

| Step              | Runner | Exit | Wall time |
| ----------------- | ------ | ---: | --------: |
| frontier-classify | Codex  |    0 |  159.880s |
| local-bulk        | Ollama |    3 |  114.150s |
| frontier-repair   | Codex  |    0 |  397.095s |
| frontier-review   | Codex  |    0 |  230.898s |
| private oracle    | local  |    0 |    0.026s |

Local resource evidence:

- `resourceSnapshotSummary.sampleCount`: `47`;
- `sampleStdoutNonEmptyCount`: `47`;
- sample `003` onward showed `qwen3-coder:30b` loaded;
- reported residency: `19 GB`;
- reported processor split: `13%/87% CPU/GPU`;
- reported context: `4096`.

Local lane failure:

```text
Prompt runner exited with code 1. Ollama runner failed: Ollama PowerShell bridge failed: Invoke-RestMethod : {"error":"model runner has unexpectedly stopped, this may be due to resource limitations"}
```

Frontier repair then completed the task. The workspace summary reported edits to
`src/app.js` and `src/test.js`, `node src/test.js` passed, and the private oracle
reported `Results: 14/14 passed` with `VERDICT: PASS`.

Decision: this is hybrid-success evidence with a local runtime resource failure.
It should be classified as `harnessFailure: true`, `modelFailure: false`, and
`resourceFailure: true` for the local lane while preserving the overall oracle pass.

## Next Measurement

Do not spend more cycles retrying the same 30B H15 local lane on this hardware.
The next useful measurement is either:

- a smaller H15-capable local candidate that can stay resident under the same
  PowerShell transport; or
- a frontier-only H15 baseline to compare wall time and frontier-call count against
  the hybrid run above.

## `HA-HR1-H15-local-devstral-small-2-001-20260508T123000Z`

This run screened `devstral-small-2:24b` as a smaller local-only H15 candidate.
The run-id timestamp label is later than the actual manifest start; the manifest
records `startedAt: 2026-05-08T11:39:30.430Z`.

Command shape:

- arm: `local-only`;
- local worker: `ollama`, `devstral-small-2:24b`, PowerShell stdin transport;
- policy label: `h15-devstral-small-2-local-screen-v1`;
- local resource sampling: `powershell.exe -NoProfile -Command "ollama ps"`;
- private oracle: `h15-api-endpoint-oracle.mjs`.

Step summary:

| Step           | Runner | Exit | Timed out | Wall time |
| -------------- | ------ | ---: | --------: | --------: |
| local-bulk     | Ollama | null |      true |  900.089s |
| private oracle | local  |    1 |     false |    0.022s |

Local resource evidence:

- `resourceSnapshotSummary.sampleCount`: `395`;
- sample stdout showed `devstral-small-2:24b` resident;
- reported residency: `16 GB`;
- reported processor: `100% GPU`;
- reported context: `4096`.

Task result:

- the local worker edited `src/app.js` and `src/test.js`;
- it added `patchContact`;
- it preserved GET, POST, and DELETE behavior;
- it did not implement required field validation;
- it added only four PATCH-related tests against the required eight or more;
- private oracle result: `9/14 passed`, `VERDICT: FAIL`.

Failed oracle checks:

- short names returned `200` instead of `400`;
- invalid emails returned `200` instead of `400`;
- invalid phones returned `200` instead of `400`;
- empty company strings returned `200` instead of `400`;
- `src/test.js` had only four PATCH-related tests.

Decision: `devstral-small-2:24b` is a better hardware fit than
`qwen3-coder:30b` on this host, but it is not promoted for H15 local-only. It
stayed resident and made partial progress, so it remains a possible bulk-draft
candidate inside a hybrid route, but only with frontier repair or a much narrower
prompt contract.

Updated next measurement:

- run a frontier-only H15 baseline for wall-time and frontier-call comparison; or
- screen a smaller code-focused model with a stricter validation/test micro-flow
  before another full H15 local-only attempt.

## `HA-HR1-H15-frontier-only-codex-001-20260508T115841Z`

This run measured the frontier-only baseline for the same H15 endpoint task.

Command shape:

- arm: `frontier-only`;
- worker: Codex through the `h15-api-endpoint-worker.flow`;
- policy label: `h15-frontier-only-baseline-v1`;
- private oracle: `h15-api-endpoint-oracle.mjs`.

Step summary:

| Step           | Runner | Exit | Timed out | Wall time |
| -------------- | ------ | ---: | --------: | --------: |
| frontier-full  | Codex  |    0 |     false |  328.917s |
| private oracle | local  |    0 |     false |    0.020s |

Oracle result:

- `Results: 14/14 passed`;
- `VERDICT: PASS`.

The frontier worker summary reported:

- implemented `patchContact(id, data)` with partial update behavior and
  field-level validation;
- preserved existing exports and GET/POST/DELETE behavior;
- added PATCH/update tests covering valid updates, invalid inputs, partial update
  preservation, and missing ID;
- ran the public structural gate and `node src/test.js`.

## H15 Route Decision After Baselines

The current H15 evidence does not support using local-first hybrid as a
cost-reduction route.

Comparison:

| Route                    | Local calls | Frontier calls | Result      | Main issue                     |
| ------------------------ | ----------: | -------------: | ----------- | ------------------------------ |
| `qwen3-coder:30b` hybrid |           1 |              3 | Oracle pass | local resource failure         |
| `devstral-small-2:24b`   |           1 |              0 | Oracle fail | timeout, incomplete validation |
| Codex frontier-only      |           0 |              1 | Oracle pass | none observed in this run      |

Decision: for H15 endpoint work, use frontier-only as the current baseline route.
Hybrid remains useful as an experiment shape, but not as the default H15 route
until a local candidate can either pass local-only or reliably reduce frontier
repair/review work.

The first useful local follow-up is now complete: run
`HA-HR1-H15-validation-only-qwen-coder-002` passed the checked-in
validation-only route with public tests `14/14`, private oracle `8/8`, zero
frontier calls, and `143.995s` wall time. That upgrades validation-only work to
positive local-screen evidence, but it does not change the full H15 route
decision.

Remaining useful local work is narrower micro-flow work, not another full H15
run:

- PATCH test-authoring only;
- one public gate repair loop with shorter timeout;
- then retest the best local model on full H15 only after the micro-flow passes.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-002`

This run screened the checked-in PATCH test-authoring micro-flow after the
validation-only route had passed.

Command shape:

- arm: `local-only`;
- local worker: `ollama`, `qwen3-coder:30b`, PowerShell stdin transport;
- policy label: `h15-patch-test-authoring-local-screen-v1`;
- private oracle: `h15-patch-test-authoring-oracle.mjs`.

Outcome:

- local step: failed with `PLR-007`;
- failure reason: `Ollama runner exceeded the action round limit (12)`;
- private oracle: `2/4`, `FAIL`;
- public tests in the generated workspace: `12/13`.

Failed oracle checks:

- missing required case: email without `@`;
- public tests failed because the generated partial-preservation test reused a
  shared contact after a prior PATCH test changed that contact's company to
  `null`.

Decision: leave H15 PATCH test-authoring not promoted at `0/1`. This is still a
useful screen because it points to a route-design fix: give exact missing-case
labels in the structural gate and tell the local worker to create fresh contacts
for PATCH tests that mutate fixture state.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-003`

This run reran the PATCH test-authoring screen after adding exact structural
gate labels and fresh-contact guidance.

Outcome:

- local step: failed with `PLR-007`;
- failure reason: `Ollama runner exceeded the action round limit (16)`;
- wall time: `580.995s`;
- private oracle: `2/4`, `FAIL`.

The route revision helped with criteria coverage: the model repaired the missing
short-phone literal and included the required email, company, null-company, and
missing-ID cases. It then rewrote the original test helper into a different
harness and treated app functions as raw-contact or throwing APIs. The H15
fixture returns `{ status, body }` response objects, so public tests failed
before the private oracle could score the tests as meaningful.

Decision: leave H15 PATCH test-authoring not promoted at `0/2`. The next route
revision should explicitly preserve the `test(name, fn)` helper and require
`result.status` / `result.body` assertions before spending another live run.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-004`

This run reran the PATCH test-authoring screen after the response-object and
test-harness preservation route hardening.

Outcome:

- local step: `0`;
- wall time: `385.343s`;
- private oracle: `4/4`, `PASS`;
- frontier calls: `0`.

The local model still required structural repair turns, but it eventually
produced `src/test.js` changes only, preserved `src/app.js`, passed public tests,
and killed the private PATCH validation mutants.

Decision: count H15 PATCH test-authoring as `1/3` positive local-screen evidence
for `qwen3-coder:30b`. This does not change the full H15 endpoint route:
frontier-only remains the baseline for full endpoint ownership.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-005`

This run checked repeatability of the passed PATCH test-authoring route.

Outcome:

- local step: `0`;
- wall time: `108.354s`;
- private oracle: `3/4`, `FAIL`;
- frontier calls: `0`.

The public flow completed, but the private oracle caught a partial-update test
quality gap. The generated test compared preserved fields against
`getContact(1).body`, which is a live object reference. Since `patchContact`
mutates that same object, a mutant that overwrites unrelated fields can still
pass the test.

Decision: keep H15 PATCH test-authoring at `1/4`, not promoted. The next prompt
revision should require primitive field snapshots before `patchContact`.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-006`

This run tested the primitive-snapshot route revision from run `005`.

Outcome:

- local step: `3`;
- wall time: `699.870s`;
- private oracle: `3/4`, `FAIL`;
- frontier calls: `0`.

The generated partial-update test used primitive expected values, but the model
rewrote the existing delete test to delete shared fixture id `1`. The existing
valid PATCH test then failed because contact `1` no longer existed, and the flow
exhausted 16 action rounds before repairing the public test failure.

Decision: keep H15 PATCH test-authoring at `1/5`, not promoted. The next prompt
revision should structurally reject `deleteContact(1)` and require delete tests
to create and delete their own contacts.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-007`

This run tested the shared-fixture delete guard from run `006`.

Outcome:

- local step: `0`;
- wall time: `254.420s`;
- private oracle: `4/4`, `PASS`;
- frontier calls: `0`.

The generated tests preserved `src/app.js`, avoided deleting fixture id `1`,
passed public tests, and killed the private PATCH validation mutants.

Decision: count H15 PATCH test-authoring as `2/6` positive local-screen
evidence. It remains a screen-only route because the latency and failure history
are not strong enough for full H15 endpoint ownership.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-008`

This run tested repeatability after the run `007` pass.

Outcome:

- local step: `0`;
- wall time: `154.862s`;
- private oracle: `4/4`, `PASS`;
- frontier calls: `0`.

The generated tests again preserved `src/app.js`, avoided deleting fixture id
`1`, passed public tests, and killed the private PATCH validation mutants.

Decision: count H15 PATCH test-authoring as `3/7` positive local-screen
evidence. Promote it only as a tests-only local screen; keep full H15 endpoint
ownership on the frontier baseline.

## `HA-HR1-H15-patch-test-authoring-qwen-coder-010-20260508T184143Z`

This run replayed the promoted PATCH test-authoring route with a hard
`--frontier-call-limit 0`.

Outcome:

- local step: `0`;
- timed out: `false`;
- wall time: `213.489s`;
- model calls: `7`;
- tokens: `20,910`;
- resource samples: `91`;
- private oracle: `4/4`, `PASS`;
- frontier calls: `0`.

The generated tests preserved `src/app.js`, changed only `src/test.js`, passed
public tests, and killed the private PATCH validation mutants. Sampled residency
showed `qwen3-coder:30b` loaded through the PowerShell transport at `19 GB`,
`13%/87% CPU/GPU`, and `4096` context.

Decision: count H15 PATCH test-authoring as `5/9` overall and `4/4` after the
shared-fixture delete guard. The route remains promoted for tests-only local
ownership, not for full H15 endpoint implementation.
