---
id: 10079
title: '[skill] Broaden ticket-intake''s Hypothesis gate to verify written claims against precedent (numerical + architectural + procedural)'
state: CLOSED
labels:
  - enhancement
  - ai
  - architecture
assignees:
  - neo-gpt
createdAt: '2026-04-19T08:43:43Z'
updatedAt: '2026-06-06T15:05:01Z'
githubUrl: 'https://github.com/neomjs/neo/issues/10079'
author: tobiu
commentsCount: 1
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-06-06T15:05:01Z'
---
# [skill] Broaden ticket-intake's Hypothesis gate to verify written claims against precedent (numerical + architectural + procedural)

## The Blind Spot — Expanded After Fifth Observation

**Originally framed** as a strengthening against hardcoded numerical thresholds in ACs. After four more observations in session `62d6f155` (2026-04-19), the underlying failure class is broader: **accepting written claims at face value without verifying against precedent code, runtime behavior, or the rule's actual grounding assumption.** Numerical thresholds are one instance; architectural descriptions, tool-description framings, self-inferred policies, and misread grounding assumptions are four more.

### Five observed instances this session

| # | Claim codified | Written source | Actual precedent / truth | Fix |
|---|---|---|---|---|
| 1 | *"Output under 100 lines for Tier 1-2"* → hard test assertion | #10033 AC checklist | Production output = 53 lines; 100 was a 2× safety margin on a 50-line estimate from a smaller-context era | PR #10078 polish commit `29fb2ac0b` — observability log replaces hard cap |
| 2 | *"how does X work → ask, file discovery → query_documents"* → codified in memory-mining anti-patterns | `ask_knowledge_base` official tool description | Empirical A/B test: ask returns synthesized answer *plus* same top-5 refs — strictly dominates query_documents | PR #10076 polish commit `e79b89a3b` + openapi.yaml commit `7eadba80f` |
| 3 | *"Universal dispatch at `ai/mcp/ToolService.mjs`"* → proposed instrumentation site for #10081 FAQ telemetry | `CodebaseOverview.md` description | Canonical precedent is `RecorderService` at per-server `services/toolService.mjs`; tobi had even cued this in the original framing (*"similar to the NL event recorder"*) | #10081 body rewritten 2026-04-19 |
| 4 | *"Architectural Pillar = fresh session"* → applied as general policy to #10035 | No written source — self-inferred from tobi's specific comment *"#9999 will require a fresh session"* | Specific user statement, not a general rule. Self-codification inverted the semantic scope: "this one requires" → "these all require" | Memory ext: `feedback_verify_written_claims_against_precedent.md` §4 (*self-inferred-policy class*) |
| 5 | *"25-turn guardrail is context-scarcity thinking"* → initially flagged as stale finding | `AGENTS.md §10.6` (correctly read; the phrase *"single task"* was in my own quote) | Thrashing-detection rule ("getting stuck" prevention, e.g., Gemini grep-looping 50×), NOT context-budget. Projected wrong grounding assumption onto a correctly-read rule | Memory ext: `feedback_verify_written_claims_against_precedent.md` §5 (*wrong-grounding-assumption class*) |

All five were caught by tobi's challenges, not by any skill firing preemptively. The common failure: treating documentation / specification / description language as spec-of-current-truth rather than evidence-of-intent, OR projecting the wrong grounding assumption onto a correctly-read rule.

## The Proposed Strengthening (expanded scope)

### Target files

- `.agent/skills/ticket-intake/references/ticket-intake-workflow.md` §1.4
- `.agent/skills/pr-review/references/pr-review-guide.md` (cross-link)
- `.agent/skills/memory-mining/references/memory-mining-protocol.md` (cross-link)

### Specific changes

1. **Enumerate the five high-risk written-claim patterns explicitly** in ticket-intake §1.4:

   > Written claims come in five high-risk classes. For each class, the Hypothesis gate requires an empirical verification step before codification:
   >
   > **(a) Hardcoded numerical thresholds** — *"under N lines"*, *"≤ N ms"*, *"at most N items"*. Verify by measuring the current value. Ask where N came from; if undocumented, prefer observability + architectural-layer assertion over hard cap.
   >
   > **(b) Architectural-description claims** — *"uses X pattern"*, *"universal dispatch at Y"*, *"similar to Z"*. Verify by reading the actual file / class / precedent in the codebase before paraphrasing it into a proposal. Especially when the user cues *"similar to X"* — that's an instruction to read X.
   >
   > **(c) Tool/API description claims** — *"use query_documents for file paths"*, *"should be your first tool for conceptual questions"*. Verify by running the tool empirically against a representative query before codifying the routing rule.
   >
   > **(d) Self-inferred policies from specific statements** — user says *"X requires fresh session"* and agent generalizes to *"all X-class requires fresh session"*. Verify by asking whether the generalization was intended, or by looking for the rule written down elsewhere. Specific statements are not universal policies.
   >
   > **(e) Wrong grounding assumption on a correctly-read rule** — read the rule accurately but project the wrong motivation onto it (e.g., "this is about token budget" when it's actually about thrashing detection). Verify by asking *"what specific failure mode does this rule prevent?"* rather than assuming based on surface framing.

2. **Add a concrete verification-action table** mapping claim class → required verification before codification:

   | Claim class | Required verification |
   |---|---|
   | (a) Numerical threshold | Measure current value + ask for derivation |
   | (b) Architectural description | `view_file` / `Glob` / `grep_search` for the named precedent |
   | (c) Tool routing / API behavior | Empirical invocation against representative input |
   | (d) Self-inferred policy | Check for the rule written down; ask whether generalization was intended |
   | (e) Wrong grounding assumption | Ask *"what specific failure mode does this rule prevent?"* before mapping to a motivation |

3. **Cross-link from pr-review** — add one line noting that self-reviews should flag any newly-codified written claim that lacks a cited verification step.

4. **Cross-link from memory-mining** — add one line noting that *"similar to X"* cues are high-priority memory-mining triggers.

### Falsifiable test of success

Future sessions receiving tickets with any of the five claim classes invoke memory-mining / precedent-reading / empirical tests before codifying — observable in PR bodies citing the verification step.

## Scope

- Purely additive edits to existing skill references
- No new skill; no refactor of existing skill internals
- ~40–60 line diff across 1–3 skill files (slightly larger than the original narrower proposal; still low-complexity)
- No production code changes

## Related

- PR #10078 — first blind-spot surface (100-line threshold, class a)
- PR #10076 — second blind-spot surface (ask vs query_documents routing, class c)
- #10081 — third blind-spot surface (universal dispatch vs per-server precedent, class b)
- #10083 — fourth and fifth blind-spot surfaces consolidated into a concrete audit fix of AGENTS.md §9
- #10075 / PR #10076 — `memory-mining` skill (adjacent to this gap)
- `feedback_challenge_numerical_thresholds_in_acs.md` — durable memory covering claim class (a)
- `feedback_verify_written_claims_against_precedent.md` — durable memory covering claim classes (b) through (e)

## Origin Session ID

`62d6f155-e57f-4279-9b59-36c9e4ecbc5e`

## Revision Note

This ticket was originally scoped narrowly to numerical thresholds (claim class a). After four more in-session observations, expanded to cover all five written-claim classes. The expanded scope preserves the original intent and absorbs what would otherwise be four follow-up tickets.

## Timeline

- 2026-04-19T08:43:44Z @tobiu added the `enhancement` label
- 2026-04-19T08:43:44Z @tobiu added the `ai` label
- 2026-04-19T09:23:33Z @tobiu changed title from **[skill] Strengthen ticket-intake's Hypothesis gate against hardcoded numerical thresholds in ACs** to **[skill] Broaden ticket-intake's Hypothesis gate to verify written claims against precedent (numerical + architectural + procedural)**
- 2026-04-19T09:42:06Z @tobiu cross-referenced by #10083
- 2026-04-19T10:25:24Z @tobiu cross-referenced by PR #10084
- 2026-04-19T10:33:36Z @tobiu cross-referenced by #10030
- 2026-05-01T08:05:43Z @neo-gpt cross-referenced by #10555
### @neo-gpt - 2026-06-06T14:33:43Z

**Input from GPT-5.5 (Codex Desktop):**

> ● ## Intake Refresh + Source-Ticket Contract Ledger
> 
> [V-B-A] I fetched the live issue conversation, checked current target paths, searched the synced issue corpus, searched live PRs for `10079`, read the current skill payloads, and derived stale-band thresholds from `.github/workflows/close-inactive-issues.yml`.
> 
> ### Ticket Reality Classification
> 
> - **Ticket age:** created `2026-04-19T08:43:43Z`, updated `2026-04-19T09:43:05Z`.
> - **Bot stale-band:** `pre-stale` by the live workflow (`days-before-issue-stale: 90`, `days-before-issue-close: 14`); no `stale` label and no `no auto close` label.
> - **Successor / duplicate evidence:** live issue has no comments; live PR search for `10079` only surfaced PR #10084, which resolves #10035 and merely cites the written-claim discipline as context. Local issue/discussion search surfaced #10079 and its incident cross-references, not a successor implementation.
> - **Current-source evidence:** `.agents/skills/ticket-intake/references/ticket-intake-workflow.md` still has only the generic Hypothesis vs Root Cause gate; `.agents/skills/pr-review/references/pr-review-guide.md` has a narrower linked-anchor accuracy check; `.agents/skills/memory-mining/references/memory-mining-protocol.md` does not yet call out `similar to X` as a high-priority precedent trigger.
> - **Path correction:** the ticket's old `.agent/...` target paths map to the live `.agents/...` tree; `.agent` does not exist.
> - **Knowledge-base note:** `ask_knowledge_base` is currently unavailable (`Failed to access collections`), so I used live GitHub, synced corpus grep, current files, and Memory Core queries as fallback evidence.
> 
> **Verdict:** `valid-as-written` after path correction to `.agents/...`, with positive ROI if the implementation stays compact and does not bloat `SKILL.md` routers.
> 
> ### Contract Ledger (T3 Source-Ticket Backfill)
> 
> | Target Surface | Source of Authority | Proposed Behavior | Fallback / Edge Case | Docs | Evidence |
> | :--- | :--- | :--- | :--- | :--- | :--- |
> | `.agents/skills/ticket-intake/references/ticket-intake-workflow.md` | #10079 live issue + current generic Hypothesis gate | Add a compact written-claim precedent gate that names the five high-risk claim classes and maps each to required verification before codification. | If a ticket claim does not match a named class, retain the existing generic V-B-A / Hypothesis vs Root Cause gate; do not create a new always-loaded router rule. | Yes, existing ticket-intake reference payload only. | Current-source `rg`/file read before edit; `git diff --check`; skill manifest lint. |
> | `.agents/skills/pr-review/references/pr-review-guide.md` | #10079 cross-link request + existing §7.4 rhetorical-drift / linked-anchor audit | Add one reviewer-side cross-link so self/peer reviews flag newly codified written claims that lack a cited verification step. | Keep as one line in the existing audit surface; do not duplicate the ticket-intake matrix inside the already-large review guide. | Yes, existing pr-review reference payload only. | Current §7.4 read; `git diff --check`; skill manifest lint. |
> | `.agents/skills/memory-mining/references/memory-mining-protocol.md` | #10079 cross-link request + current architectural-claim trigger | Add one line that `similar to X` / precedent-comparison cues are high-priority memory-mining or precedent-reading triggers. | Preserve current tool-order rules; do not make every routine comparison a Memory Core query when direct current-source precedent reading is the right falsifier. | Yes, existing memory-mining reference payload only. | Current protocol read; `git diff --check`; skill manifest lint. |
> 
> ### Load-Effect Placement
> 
> This is a skill-payload change, not a router change. The intended load effect is conditional: agents load these payloads only when `/ticket-intake`, `/pr-review`, or `/memory-mining` fires. No `SKILL.md`, `AGENTS.md`, or harness turn-loaded file should change for this ticket.

- 2026-06-06T14:33:52Z @neo-gpt assigned to @neo-gpt
- 2026-06-06T14:34:22Z @neo-gpt added the `architecture` label
- 2026-06-06T14:40:13Z @neo-gpt cross-referenced by PR #12637
- 2026-06-06T15:05:01Z @tobiu referenced in commit `c0709e6` - "feat(agentos): add written-claim precedent gate (#10079) (#12637)"
- 2026-06-06T15:05:01Z @tobiu closed this issue

