Pattern families → OrchestKit surfaces
Green = strong implementation · amber = partial · red = missing. Click tabs above for per-family deep dives.
flowchart LR classDef good fill:#1d3325,stroke:#3fb950,color:#e6edf3 classDef warn fill:#332a14,stroke:#d29922,color:#e6edf3 classDef bad fill:#3a1d1d,stroke:#f85149,color:#e6edf3 classDef fam fill:#2a1d3a,stroke:#d2a8ff,color:#e6edf3,font-weight:bold F1["🔁 Self-correction loops
6.2/10"]:::fam F2["⚖️ Verifier sub-agents
7.0/10"]:::fam F3["🧠 Memory outer loop
4.4/10"]:::fam F1 --> A1["prd-to-goal 9/10
PRD → /goal assertions"]:::good F1 --> A2["brainstorm iterative-opt 8.5/10
autoresearch metric loop"]:::good F1 --> A3["cover heal loop 7/10
ci-sentinel 7/10"]:::warn F1 --> A4["rubric-file convention
grader-in-loop · CMA Outcomes"]:::bad F2 --> B1["verify 6-7 fork graders
composite GATES merge"]:::good F2 --> B2["adversarial refutation
blind refuters + ledger + quorum"]:::good F2 --> B3["assess advisory-only
no dimension blockers"]:::warn F2 --> B4["machine-readable
rubric contract"]:::bad F3 --> C1["investigate 7/10
dream · staleness cron · lint"]:::good F3 --> C2["consult 4/10
only 2/6 auto-inject"]:::warn F3 --> C3["VERIFY loop never closes
distill fragmented · journal unbuilt"]:::bad
✅ Strongest: independent-context verification
verify/review-pr/assess already do what Lance recommends — grading in fork contexts, blind adversarial refuters, citation re-verification, anti-sycophancy. The bones of "Outcomes" exist.
⚠️ Partial: loops exist but each is bespoke
prd-to-goal is a reference rubric-as-assertions implementation, but every loop (cover ×3, brainstorm stuck-5, ci-sentinel hourly) hard-codes its own convergence logic. No shared rubric file, no grader auditing /goal assertions.
🔴 Weakest: memory stops at step 2–3
Per Lance's taxonomy OrchestKit memory behaves like "Opus 4.7": good investigation, low verification coverage, fragmented distillation (recent-decisions.md corrupted), mostly-manual consultation. The Fable 5 differentiator — closing fail→…→consult — is the open prize.
🔁 Family 1 — Self-correction loops · 6.2/10
"Let Claude run, collect feedback via the goal or rubric, self-correct, and proceed until satisfied."
flowchart LR classDef have fill:#1d3325,stroke:#3fb950,color:#e6edf3 classDef miss fill:#3a1d1d,stroke:#f85149,color:#e6edf3,stroke-dasharray:5 4 R["📋 Rubric / Goal"]:::have --> RUN["▶️ Run task"]:::have RUN --> FB["📊 Collect feedback
tests · metrics · CI"]:::have FB --> SC["🔧 Self-correct"]:::have SC --> RUN FB --> DONE{"rubric
satisfied?"}:::have DONE -- yes --> STOP["⏹️ stop"]:::have RF["📄 rubric.json file contract
(user-editable, cross-skill)"]:::miss -. missing .-> R GR["🧑⚖️ grader audits assertions
on /goal timeout"]:::miss -. missing .-> DONE CMA["☁️ CMA Outcomes integration"]:::miss -. missing .-> STOP
| Surface | Loop mechanism | Score | Missing |
|---|---|---|---|
| prd-to-goal | PRD → /goal until observable assertions | 9.0 | assertion-quality grader |
| brainstorm iterative-opt | baseline → ideate → measure → keep/discard → stuck-5 | 8.5 | rubric file; underused |
| audit-full | /goal until findings.critical == 0 | 8.0 | configurable severity ladder |
| ScheduleWakeup | cache-aware poll → decide → reschedule | 7.5 | no rubric, imperative only |
| cover (healer) | generate → run → heal failures ×3 | 7.0 | hard-coded 3; no test-quality rubric |
| ci-sentinel | hourly classify-failing-PRs, propose-don't-apply | 7.0 | no auto-learning from approved fixes |
| verify / assess | rubrics exist but grade once — no loop | 5.0 | no "score < target → improve → re-run" |
⚖️ Family 2 — Verifier sub-agents · 7.0/10
"A verifier sub-agent tends to outperform self-critique, because grading is done in an independent context window." The gap is authority: who can actually block stop?
flowchart TB classDef good fill:#1d3325,stroke:#3fb950,color:#e6edf3 classDef warn fill:#332a14,stroke:#d29922,color:#e6edf3 classDef bad fill:#3a1d1d,stroke:#f85149,color:#e6edf3 W["🛠️ Producer (lead context)"] --> V1["verify: 6-7 fork graders
8 dims, composite under 6.0 BLOCKS ✅"]:::good W --> V2["review-pr: blind refuters
quorum + ledger + no-auto-flip ✅"]:::good W --> V3["assess: refuters but
ADVISORY ONLY ⚠️"]:::warn W --> V4["bare-eval: fresh-context judge
primitive, no verdict surface ⚠️"]:::warn V1 --> GATE{"stop-gate"} V2 --> GATE V3 -. no authority .-> GATE GATE --> X1["❌ no dimension-level blockers
(security 3.2 hides in passing composite)"]:::bad GATE --> X2["❌ no Outcomes-style
grader-must-approve-stop"]:::bad
| Surface | Independent context | Refutation | Stop-gating |
|---|---|---|---|
| verify | ✅ fork ×6-7 | — | YES — composite gates merge |
| review-pr | ✅ fork, no team | ✅ blind, quorum | YES — CRITICAL → request-changes |
| assess | ✅ fork ×4 | ✅ Phase 2.5 | NO — advisory only |
| cover | ✅ worktree/tier | — | implicit (flag after 3 heals) |
| bare-eval / eval-runner | ✅ --bare fresh | — | NO — pipeline primitive |
| brainstorm devil's advocate | ✅ agents | partial | NO — ranking only |
🧠 Family 3 — Memory progression · 4.4/10
Fable 5 completes fail → investigate → verify → distill → consult (verification coverage up to 73%). OrchestKit currently exits around step 2–3 — the "Opus 4.7" profile.
flowchart LR classDef s3 fill:#3a1d1d,stroke:#f85149,color:#e6edf3 classDef s7 fill:#1d3325,stroke:#3fb950,color:#e6edf3 classDef s5 fill:#332a14,stroke:#d29922,color:#e6edf3 classDef s4 fill:#332a14,stroke:#d29922,color:#e6edf3 FAIL["1️⃣ FAIL · 3/10
remember skill, handoffs
❌ no 'ignored-advice' tracking"]:::s3 INV["2️⃣ INVESTIGATE · 7/10
dream refs, staleness cron,
memory-lint, validator"]:::s7 VER["3️⃣ VERIFY · 5/10
staleness classified BUT
reports never consumed"]:::s5 DIS["4️⃣ DISTILL · 3/10
dream dedup exists BUT
recent-decisions.md corrupted,
experiment journal unbuilt"]:::s3 CON["5️⃣ CONSULT · 4/10
2/6 auto-inject; brainstorm
prints a reminder, not results"]:::s4 FAIL --> INV --> VER --> DIS --> CON CON -. outer loop never closes .-> FAIL
Gap M1 — VERIFY never closes the loop
Nightly staleness reports have zero consumers; no signal records whether a consulted (or ignored) memory changed an outcome. This is the exact step Lance identifies as the Fable 5 differentiator.
Gap M2 — DISTILL fragmented + corrupted index
recent-decisions.md (the distilled-rules index) contains mangled truncated entries and has no owning write mechanism. Brainstorm's experiment journal is referenced in phase-workflow.md but the TSV was never implemented.
Gap M3 — CONSULT is manual
priorDecisionsLoader injects "IMPORTANT: search memory" instead of running the search and injecting results. Only session-handoff + memory-lint auto-inject.
🕳️ Gap heatmap — ranked by (impact × cross-family reach) / effort
Note how the rubric-file contract and stop-gating grader each appear in two families — one fix, double coverage.
| # | Gap | Family | Severity | Effort | Fix sketch |
|---|---|---|---|---|---|
| 1 | No machine-readable rubric contract | 🔁 + ⚖️ | HIGH | 3-4 d | ork-rubric/1.0 schema; backfill verify/assess/review-pr/prd-to-goal |
| 2 | assess advisory-only — no stop-gating | ⚖️ | HIGH | 2-3 d | emit assess-verdict.json; gate /ork:implement at <5.5 |
| 3 | Memory VERIFY loop never closes | 🧠 | HIGH | 1 wk | consume staleness reports in CI; record consult→outcome signals |
| 4 | recent-decisions.md corrupted; journal unbuilt | 🧠 | HIGH | 2-3 d | dream owns regeneration; implement brainstorm TSV journal |
| 5 | No grader-in-the-loop for /goal | 🔁 | MED | 2 d | post-timeout grader audits assertion quality |
| 6 | verify lacks dimension-level blockers | ⚖️ | MED | 1-2 d | security <4.0 always blocks regardless of composite |
| 7 | CONSULT manual (4/6 surfaces) | 🧠 | MED | 2-3 d | priorDecisionsLoader runs search, injects results (relevance-gated) |
| 8 | No cross-skill loop primitive | 🔁 | MED | ? | likely a chain-patterns reference, NOT a skill ("/workflows: use don't wrap") |
| 9 | No CMA / Outcomes integration | 🔁 + ⚖️ | LOW* | n/a | watch — hosted product surface, plugin can't reach it yet |
* LOW for the plugin today; HIGH strategically if CMA exposes an API the plugin can target.
🗺️ Recommended sequencing
A → B → (C reassessed). Approach A closes the #1 gap of two families with one contract.
flowchart TB classDef rec fill:#2a1d3a,stroke:#d2a8ff,color:#e6edf3,font-weight:bold classDef nor fill:#161b22,stroke:#79c0ff,color:#e6edf3 classDef opt fill:#161b22,stroke:#8b949e,color:#8b949e,stroke-dasharray:5 4 A["🅰️ Rubric contract + stop-gating grader · 8.3
ork-rubric/1.0 schema · assess gates implement
verify dimension blockers · /goal assertion grader
≈ 1 week"]:::rec B["🅱️ Close the memory VERIFY loop · 7.8
consume staleness reports · consult→outcome signals
fix recent-decisions.md · experiment journal TSV
auto-inject prior decisions · ≈ 1 week"]:::nor C["🅲 loop-until-rubric primitive / CMA parity · 6.5
⚠️ devil's advocate: '/workflows — use, don't wrap'
after A this may reduce to a documented pattern"]:::opt A -->|"rubric schema unlocks
verification instrumentation"| B A -->|"grader gate makes C
mostly free"| C B --> DONE["🏁 OrchestKit completes the progression
= 'Fable 5 memory profile' per Lance's taxonomy"] C -. reassess .-> DONE
🅰️ Rubric contract + gating — composite 8.3 · RECOMMENDED
One JSON schema (dimensions, weights, min_pass, min_blocker, bands) shared by verify/assess/review-pr/prd-to-goal. assess emits a verdict file that gates /ork:implement; verify gains dimension blockers; /goal gains a post-timeout assertion grader.
🅱️ Memory verify loop — composite 7.8
Targets the weakest family and the Fable-5-specific differentiator. Weekly job diffs staleness reports + warns on PRs touching stale-memory files; consult events recorded; dream regenerates recent-decisions.md.
🅲 Loop primitive — composite 6.5 · DEFER
A generic loop-until-rubric harness risks wrapping what CC-native /goal + Workflow already provide (prior decision: "use /workflows, don't wrap"). Build A first; C likely becomes a chain-patterns reference.
🎛️ Goal Builder — turn this analysis into a /goal line
Pick scope and effort; copy the generated kickoff prompt + /goal line into Claude Code.