Sibyl 1.5Trust It

At a glance

Four workstreams in priority order, with a hard ceiling of $435 in inference spend and a likely spend nearer $300. Sibyl is an open-source project, so the plan buys evidence as cheaply as the statistics allow. The scale and migration work runs on the devbox and free public-repo CI, and engineering time is not counted. The release ships when the trust, scale, spend-guard and migration-rehearsal gates pass. The study runs alongside and decides only what the release note may claim. Every gate figure in this document is a proposed acceptance criterion, not a measurement, and every spend figure is a ceiling awaiting approval.

Decision

1.5 is a trust release. 1.4 proved that the dream cycle runs end to end with receipts, and the September 25 cleanup fixed the reasons it learned the wrong things. What a user meets first, though, is not consolidation. They meet recall that crosses projects, a delete that returns 500, a correction they can't undo from the CLI, and a graph page that takes a minute and then fails. 1.5 fixes those, makes the system hold up at the size real graphs reach, and delivers one real team migration.

The learning proof stays a first-class workstream, but it doesn't gate the release. The study is staged behind cheap gates, so money is spent only once the pipeline can plausibly show an effect, and the whole plan's inference spend is capped at $435. If the sealed study passes the 1.4 claim gate, 1.5 carries a bounded claim. If it doesn't, the release note reports the measured result and 1.5 still ships.

1.5 does not: activate Jev in production, claim general or autonomous learning, chase leaderboard numbers, polish graph visuals beyond the capacity fix, expand the archive or memoryfield surfaces, or build team workflow features beyond migration. Proposals in those areas go on the 1.6 list.

Where 1.4 left us

The learning mechanism now keeps what matters

The September 20 re-probe showed memory of any kind lifting the three headroom tasks, but native memory did not beat raw retrieval (native 12/15, raw 13/15). The cause was consolidation: 33 of the 34 procedures it promoted were generic repair choreography. Four PRs merged on September 25 address that.

The intervention probe (task b7bc8bf4) gives the only native-over-raw signal so far: 14/18 against 9/20 on cdn-batch-map, a paired +6 of 18 at p = 0.11. That clears the pre-stated +4 threshold but isn't significant. On the 16 repetitions where both arms finished, removing the derived items cost only 2 of those wins (11/18 overall against 14/18), a family summary adds nothing over raw, and a task with opaque keys shows no negative transfer (10/10 in both arms). One more finding from the probe: the promoted pattern itself was excluded from native recall as out of scope, and the mechanism reached the solver only through two raw candidate rows (task 3ed9000e).

What a user meets first

The September 22 demo prep and the proof work surfaced trust defects, and every one of them is still present on main. Each is small and reproduced. The gap that matters most is in process: PR CI never runs the :check bundle, so a delegated-scope regression from PR 525 shipped through 11 days of green PRs and surfaced only at the 1.4.0 dry cut.

The system slows as it grows

On the real 10.5K-entity graph, /graph spins for about 70 seconds and then fails with a 500, starving other pages in the same tab. A long-lived Surreal process reached 33 GB resident, with plain selects taking 14.8 seconds. Sustained ingest degrades from about 10 seconds to 100 to 700 seconds per capture once the entity table deepens. These also contaminate every latency number the public benchmarks would report.

Public benchmarks have no current anchor

LongMemEval-S sits at 26.2% because the adapter truncates every session at 3,985 characters and never calls compile_context. V2 Web Small's 23.3% ran with notes off and isn't comparable to the notes-on 30.38% anchor. The 80% and 60% targets in the 1.4 plan were never measured.

Spend so far

The 1.4 campaign spent about $463: roughly $385 on memory calls and $8.50 on solver attempts through September 19, then $69.47 across the headroom screen, re-probe, three scoped cycles and the intervention.

Release rules

1. Trust gates run on every PR. Nothing that affects scope, authorization or lifecycle merges without the gate bundle having run on its head.
2. Scale fixes grow capacity. Truncating results, lowering concurrency or skipping work doesn't count as a fix. A temporary mitigation needs a name, a tracking task and a stated path back.
3. Every paid run sits behind a gate and a guard. Each spend stage unlocks only when the previous gate passes. No product default ships with unbounded model spend.
4. Claims need sealed evidence. Design, analysis and exclusions are frozen before unsealing. Failures stay in the denominators, and nothing is rerun selectively.
5. Verification is independent. Every non-trivial PR gets an adversarial verifier that did not write it, plus root moon run :lint and :typecheck before push.

Goal: what a user stores stays in the scope they stored it in, and what they correct, delete or restore behaves the way the CLI and API say it will. Approach: fix the CI gap first, so every later fix lands under the gates. Then generate one authorization matrix that covers every role and scope across MCP, CLI and REST, and fix defects until it's green. The matrix extends the existing team-scope gate, which today seeds only private, team, project and delegated fixtures and says nothing about SHARED or ORGANIZATION.

Acceptance

Goal: the dream cycle turns related experience into procedures that reach the next agent, at a bounded cost, and one sealed study measures whether that beats simpler memory. Approach: finish the product fixes first, then build task families with real headroom, then spend in stages.

Product fixes, before any study spend

Task families with headroom

Only cdn-batch-map has real native-versus-raw room today; stock-adjustment-log and department-unit-budget saturate near raw. The study needs harder families before it needs a bigger n. Keep cdn-batch-map and add two families with different mechanisms, for example retry with idempotency keys, and revision-sensitive authorization. Each family needs:

• At least 25 instances: 5 for development screening and 20 sealed.
• Cases where applying the remembered procedure would be wrong, so negative transfer is measurable.
• Task validity verified independently of any memory arm, with public checks the solver can run and hidden checks it can't.
• A learning corpus of episodes that captures the family's mechanism, generated the way screen48's was.

The saturated screen48 tasks become regression controls.

Staged spend

The study is sized to what the statistics need. Uncertainty is clustered by instance, so once each instance has a few repetitions, more repetitions add little power; new instances add a lot. The plan keeps 20 sealed instances per family and lets the screen's variance set the repetitions. At the intervention's measured $0.19 per attempt ($29.98 for 160 cells), 1,300 attempts cost about $250, and the $300 cap leaves room for harder tasks running longer. The interim look means about half the time the study stops at half the cost.

Evaluation contract

What the release note says, by outcome

Goal: the graph page, recall and ingest stay responsive at ten times today's real graph, with concurrent clients. Approach: treat the four symptoms as one capacity investigation on the devbox against native Surreal (local OrbStack has wedged four times under this kind of load). Name the contended resource for each symptom before changing code. The work runs on the devbox and free public-repo CI, so it adds no inference spend.

Acceptance

Goal: any team member can move their personal Sibyl into a shared org server with one command, and one real team does it end to end. Both tasks have waited at high priority since September 2. An archive import makes no model calls, so the move adds no inference spend.

Acceptance

Side lanes with a set position

What users get

• Recall and web pages scoped to the current project by default, with cross-project access as a visible choice.
• Delete, restore, hide and redact from the CLI, with previews, and no forever-buffered writes.
• SHARED memories that behave as documented.
• A graph page that renders in seconds on a real graph without starving the rest of the app.
• Ingest and recall that stay fast as memory grows.
• A dream cycle that groups related experience across pages and keeps the mechanism in what it promotes.
• Opus memory models sized to their window, with spend bounded per run.
• sibyl migrate to-team for moving personal memory into a team server.
• A health check that tells the truth about storage.

Release boundary

The cut follows the 1.4 mechanics: bump VERSION, run moon run sync-versions for the 11 static pins, do a release dry run (it caught the delegated-scope regression in 1.4.0), then cut. The tag, publish and release steps each need your go-ahead.

Sequence

Each phase ends on a gate rather than a date. From P1 on, workstreams two and three run side by side, with the study on the devbox and the capacity work on a separate native Surreal stack.

1. P0
   Land the cleanup and close the CI gap

   Merge #661. Wire :check into PR CI (c5ecc5b4). Land the dream spend guard and refund (bb650b6e, a5b6cc73).

   Exit: main carries the whole partition stack, trust gates run on every PR, and no dream run can spend without a budget.
2. P1
   Trust fixes, recall defects, task families

   Default project scoping, the SHARED decision, CLI lifecycle, and the authorization matrix. Fix 3ed9000e, f1093a5a, fe47bb18 and 258af557. Build the two new task families and run the $30 headroom screen. Start the residency bisect on the devbox.

   Exit: the authorization matrix is green, at least two families show headroom, and the capacity bottlenecks are named.
3. P2
   Canary, cycle, study, capacity fixes

   $20 canary, then the $75 fresh cycle, the power preflight and the sealed study with its interim look. Capacity fixes land against the load harness.

   Exit: the canary has run, the sealed study is unsealed or stopped, and the scale gates pass. The study verdict feeds the release note whenever it lands and never holds P3.
4. P3
   Migration and release

   sibyl migrate to-team, two rehearsals, then the release dry run and the 1.5.0 cut. The team pilot starts on the RC.

   Exit: :check green at the RC, rehearsals reconciled, and your go-ahead for the tag and publish.

Dependencies that set the order

Spend and scope controls

• The harness cost guard stays armed on every paid run and counts billed-but-unmeasured attempts.
• A stage that would overrun its cap stops and reports. It never borrows from the next stage.
• If the canary stops the study, total spend is about $60. If the interim look stops it for futility, total spend is about $285.
• New scope goes on the 1.6 list unless it fixes a defect that blocks a 1.5 gate.

Risk register

Decisions taken

These are settled and the plan builds on them. Each has a line of reasoning so the choice can be revisited if the reasoning stops holding. Say so if one is wrong, and the plan changes.

First execution slice

The first PRs, in order. Each one goes through independent verification and root gates before merge.

In parallel from day one: the residency bisect on the devbox (8a56f5c7), since it gates the scale fixes and needs no merges.

Evidence this plan leans on