{
  "_about": "fak position rows for the 'decoding' group. dim_id links each to a _taxonomy.json dimension.",
  "rows": [
    {
      "id": "speculative-decoding-throughput-effect",
      "dim_id": "speculative-decoding-throughput-effect",
      "axis": "Speculative decoding throughput effect across batch sizes (does it survive at high concurrency?)",
      "category": "speculative-decoding",
      "competitor": "EAGLE-3; EAGLE-2 (regresses at batch); vLLM / SGLang / TensorRT-LLM speculative backends",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://docs.sglang.ai/backend/speculative_decoding.html (2025)",
      "fak_value": null,
      "unit": "× throughput at batch",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No speculative-decoding throughput-vs-batch number in BENCHMARK-AUTHORITY.md. CLAIMS.md L28 references speculative-decoding only as inverted PRIOR ART for batch ADJUDICATION (set-shape decision equals serial in one pass), not as a generation-speed spec-decode backend; no acceptance-rate or batch-scaling tok/s is measured.",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE for fak's shipped engine: fak borrows the speculative-decoding PATTERN for batched adjudication (verify a set of decisions in one pass) but does not ship a draft/target speculative-decoding generation backend, so there is no EAGLE-style acceptance-rate or batch-scaling throughput number. No fak value exists; named as out-of-scope rather than a measured gap.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "speculative-decoding-itl",
      "dim_id": "speculative-decoding-itl",
      "axis": "Speculative decoding effect on ITL (acceptance-rate-bound)",
      "category": "speculative-decoding",
      "competitor": "EAGLE-3; Medusa-2; TensorRT-LLM / vLLM speculative decoding",
      "competitor_class": "sota",
      "competitor_value": 3.5,
      "competitor_source": "https://www.spheron.network/blog/eagle-3-speculative-decoding-gpu-cloud/ (2026-01)",
      "fak_value": null,
      "unit": "x decode token-rate / acceptance rate",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md: polymodel AcceptGreedy/AcceptTree + single-pass batched/tree verify (internal/spec, #533) are SHIPPED as the accept DECISION + bit-exact rollback, proven token-identical to plain greedy - but explicitly 'no GPU, so no tokens/sec', off mainline (NOT in defconfig, FAK_POLYMODEL default off). No acceptance-rate or ITL-speedup number is committed.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP (mechanism built, number unmeasured). fak has the correctness machinery for speculative decoding (Medusa/EAGLE-2/SpecInfer-style tree verify, bit-exact via KVCache.Evict) and the closed-form EffectiveTokensPerVerify model, but no measured acceptance rate or ITL speedup - only the arithmetic, pending the bench harness (#535). EAGLE-3 reports ~0.80-0.88 acceptance / ~3-4x decode-rate on H100/H200. fak claims no number here; no-claim until the lane runs on a GPU.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "spec-decode-speedup",
      "dim_id": "spec-decode-speedup",
      "axis": "Speculative-decoding wall-clock speedup (single-stream / low-concurrency)",
      "category": "speculative-decoding",
      "competitor": "EAGLE-3 (SafeAILab); vLLM; SGLang; P-EAGLE (vLLM)",
      "competitor_class": "sota",
      "competitor_value": 4.79,
      "competitor_source": "https://github.com/SafeAILab/EAGLE (2025-09)",
      "fak_value": null,
      "unit": "x wall-clock speedup",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "projected",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md In-kernel model: internal/polymodel AcceptGreedy/AcceptTree + internal/spec SpeculativeGreedy/SpeculativeTree + model.VerifyForward (rungs #533); honest fence states it runs on the CPU synthetic PreNorm regime, NO GPU, so NO tokens/sec — the speedup is only the closed-form EffectiveTokensPerVerify arithmetic; a measured number needs bench harness #535. Off mainline (FAK_POLYMODEL default off).",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP fak should measure. fak has shipped speculative-decode MECHANISM (greedy + tree verify, single-pass batched), proven token-identical to greedy — but ZERO measured wall-clock speedup: no GPU path, no real draft model, synthetic CPU only, off-mainline by construction. EAGLE-3's up-to-4.79x is a measured wall-clock on LLaMA-3.3-70B; fak has no comparable number, so no verdict is honest. The honest bench is gated on #535.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "acceptance-length-rate",
      "dim_id": "acceptance-length-rate",
      "axis": "Draft acceptance length / acceptance rate (mean accepted tokens per verify step)",
      "category": "speculative-decoding",
      "competitor": "EAGLE-3; DeepSeek-V3 MTP; Medusa",
      "competitor_class": "sota",
      "competitor_value": 0.85,
      "competitor_source": "https://arxiv.org/pdf/2412.19437 (2024-12)",
      "fak_value": null,
      "unit": "mean accepted tokens / verify step",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "projected",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md: polymodel.EffectiveTokensPerVerify is a geometric-series speedup MODEL (closed-form arithmetic), not an empirical acceptance rate; AcceptGreedy/AcceptTree KEEP/EVICT counts are proven conserved against the bit-exact KVCache.Evict rollback but are exercised on synthetic/adversarial drafts (cmd/polymodelbench forces a rollback every round), not measured on a real draft model against a real target.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP fak should measure. fak proves the accept/rollback ACCOUNTING is correct (KEEP+EVICT conservation, lossless), but reports no empirical tau / acceptance rate from a real draft+target pair. EAGLE-class ~0.6-0.8 per-token and DeepSeek-V3 MTP ~85-90% 2nd-token are measured on real models; fak's only number is a parameterized model, so it cannot claim an acceptance rate. Competitor value shown is DeepSeek-V3 MTP 2nd-token acceptance.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "draft-method-coverage",
      "dim_id": "draft-method-coverage",
      "axis": "Breadth of speculative methods supported (draft-model, EAGLE/-2/-3, Medusa, MTP, lookahead, n-gram/prompt-lookup)",
      "category": "speculative-decoding",
      "competitor": "TensorRT-LLM; vLLM; SGLang",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://nvidia.github.io/TensorRT-LLM/advanced/speculative-decoding.html (2025-12)",
      "fak_value": null,
      "unit": "speculative methods supported (live)",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "projected",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md In-kernel model: internal/polymodel ships AcceptGreedy (chain) + AcceptTree (the token-TREE generalization explicitly mapped to Medusa/EAGLE-2/SpecInfer) + PickDrafter ensemble selection, with single-pass batched + tree-attention verify (model.VerifyForward, #533). But it is the policy/accounting brain + a CPU synthetic verify only — off mainline (not in defconfig, FAK_POLYMODEL default off); no draft-target with a real draft model, no trained EAGLE/-2/-3 head, no native MTP weights, no n-gram/prompt-lookup.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP. fak's tree-verify mechanism is genuinely EAGLE-2/Medusa/SpecInfer-shaped and lossless, but as an unwired CPU-synthetic accounting layer it supports no method end-to-end on the live serving path. TensorRT-LLM/vLLM ship 4-6 production methods (draft-target, Medusa, EAGLE/-3, MTP, lookahead, n-gram). fak claims no breadth; naming this prevents the shipped accept-core from being read as method parity.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "ngram-prompt-lookup",
      "dim_id": "ngram-prompt-lookup",
      "axis": "Retrieval / prompt-lookup (n-gram) drafting for input-grounded tasks",
      "category": "speculative-decoding",
      "competitor": "vLLM (ngram / prompt-lookup); TensorRT-LLM (Lookahead)",
      "competitor_class": "sota",
      "competitor_value": 2.8,
      "competitor_source": "https://blog.vllm.ai/2024/10/17/spec-decode.html (2024-10)",
      "fak_value": null,
      "unit": "x speedup on input-grounded tasks",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No fak evidence: CLAIMS.md / BENCHMARK-AUTHORITY.md contain no n-gram, prompt-lookup, or retrieval-drafting claim. fak's PickDrafter selects among drafters but no n-gram/prompt-lookup drafter exists.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP fak should measure (it is adjacent to fak's thesis). Prompt-lookup is input-grounded reuse — the SAME family as fak's cross-agent prefix reuse — and needs no draft model, so it is a natural fit for a reuse kernel and worth a measured row. fak has zero evidence today. Competitor value is the up-to-2.8x summarization figure.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "spec-decode-lossless-correctness",
      "dim_id": "spec-decode-lossless-correctness",
      "axis": "Output-distribution losslessness of speculation (greedy and stochastic)",
      "category": "speculative-decoding",
      "competitor": "vLLM; speculative sampling (Leviathan et al.)",
      "competitor_class": "reference-oracle",
      "competitor_value": null,
      "competitor_source": "https://docs.vllm.ai/en/latest/features/speculative_decoding/ (2025-06)",
      "fak_value": null,
      "unit": "token-identity vs greedy (max|delta| = 0 path)",
      "higher_is_better": true,
      "verdict": "parity",
      "status": "shipped",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md In-kernel model (#533): spec.SpeculativeGreedy verifies a k-token draft in ONE VerifyForward pass and stays token-identical to plain greedy; spec.VerifyTree/SpeculativeTree commit only the accepted path and are token-identical to plain greedy decode. Witnesses: TestSpeculativeTreeLosslessGreedyPath (21/63 accepted, distractors rejected by the tree mask), TestSpeculativeTreeLosslessArbitrary (lossless regardless of drafter quality), TestVerifyTreeRewindsAndCommitsCleanly (target cache byte-exact to a greedy session after the round), TestVerifyForwardChainMatchesSerial. go test ./internal/model ./internal/spec.",
      "apples_to_apples": true,
      "comparison_note": "Correctness-oracle => parity (a losslessness property is matched, never beaten). fak's speculative path is proven token-identical to plain greedy decode (the accepted path is bit-exact via KVCache.Evict rollback), the GREEDY-equivalence guarantee. HONEST FENCE vs the SOTA bar: this is proven for GREEDY only on the CPU synthetic PreNorm model; fak has NOT shown distribution-preserving rejection sampling under STOCHASTIC (temperature>0) decoding the way vLLM's implementation does — fak's in-kernel decode is greedy-only, so the sampled-decoding half of the losslessness bar is unaddressed. The greedy half is at parity; the stochastic half is unbuilt.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "spec-decode-batch-scaling",
      "dim_id": "spec-decode-batch-scaling",
      "axis": "Speculative-decoding behavior under high concurrency / large batch (break-even)",
      "category": "speculative-decoding",
      "competitor": "MagicDec; vLLM continuous batching; SGLang",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://arxiv.org/html/2408.11049v1 (2024-08)",
      "fak_value": null,
      "unit": "break-even concurrency / speedup at batch",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md In-kernel model: the polymodel decode lane is explicitly SERIAL (the at-most-one-model-decodes-per-step invariant is asserted); CLAIMS Engine labels continuous-batching [SIMULATED] (read-only telemetry, not on the live serving path). No batch-scaling / break-even measurement exists.",
      "apples_to_apples": false,
      "comparison_note": "OUT OF SCOPE for a reuse kernel, and a real gap relative to a full serving stack. Spec-decode-under-high-concurrency (MagicDec fixed-window draft restoring speedup at large batch) presupposes continuous batching, which fak does not have on the live path (continuous-batching is [SIMULATED]; the decode lane is serial by construction). fak makes no batch-scaling claim. Reuses the existing 'continuous-batching' / 'served-throughput-vs-sglang' gap framing.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "constrained-decode-per-token-overhead",
      "dim_id": "constrained-decode-per-token-overhead",
      "axis": "Constrained/guided decoding per-token overhead (JSON-schema, regex, grammar/CFG)",
      "category": "structured-output",
      "competitor": "XGrammar; LLGuidance; SGLang; vLLM",
      "competitor_class": "sota",
      "competitor_value": 40,
      "competitor_source": "https://blog.squeezebits.com/70642 (2025-03)",
      "fak_value": null,
      "unit": "microseconds / token mask overhead",
      "higher_is_better": false,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md Pre-flight ladder: '[STUB] Decode-time logit-mask (grammar-constrained generation) — the strongest form (never-emit a malformed call) — requires owning the decode loop; not in v0.1.' fak's grammar rung is a post-hoc positional->named TRANSFORM at the syscall boundary, not a per-token decode-time bitmask FSM.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP. Constrained/guided decoding (JSON-schema/regex/CFG bitmask per token) is explicitly [STUB] in fak — fak owns its decode loop in the in-kernel model but has not built the logit-mask. XGrammar's <40 microsecond/token is the bar; fak has no per-token mask overhead to report. Competitor value is XGrammar's <40us/token.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "grammar-compilation-latency",
      "dim_id": "grammar-compilation-latency",
      "axis": "Grammar/schema compilation latency and cache (cold-start for dynamic agentic schemas)",
      "category": "structured-output",
      "competitor": "XGrammar-2; XGrammar; LLGuidance/Guidance",
      "competitor_class": "sota",
      "competitor_value": 10,
      "competitor_source": "https://blog.mlc.ai/2026/05/04/xgrammar-2-fast-customizable-structured-generation (2026-05)",
      "fak_value": null,
      "unit": "ms grammar/schema compile",
      "higher_is_better": false,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md Pre-flight ladder grammar rung: 'content-addressed grammar dedup' exists for the TOOL-INVOCATION grammar (positional->named repair), but there is no committed compilation-latency number, and the decode-time grammar FSM is [STUB] (not built). No XGrammar/Earley-style mask-cache compile metric exists.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP, partial-scope. fak has a content-addressed dedup CACHE for its tool-arg grammar (so a repeated grammar is not re-parsed), conceptually adjacent to XGrammar's cross-grammar cache, but it is a tool-call repair grammar, not a CFG/JSON-schema decode mask, and carries no committed compile-latency number. XGrammar-2's ~10ms (vs >1000ms) is the bar; fak makes no measured claim. Competitor value is XGrammar-2 ~10ms.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "structured-output-validity-vs-quality",
      "dim_id": "structured-output-validity-vs-quality",
      "axis": "Structured-output validity vs reasoning-quality tax (constraint-induced degradation)",
      "category": "structured-output",
      "competitor": "JSONSchemaBench (Outlines/XGrammar/Guidance/llama.cpp/OpenAI/Gemini); OpenAI Structured Outputs",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://arxiv.org/pdf/2501.10868 (2025-01)",
      "fak_value": null,
      "unit": "schema validity / coverage %",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No fak evidence: fak runs no constrained generation ([STUB] decode-time logit-mask) and has no JSONSchemaBench run. fak's answershape witness grades degeneration/repetition shape, not schema validity, and its grammar rung repairs tool-arg arity, not output schemas.",
      "apples_to_apples": false,
      "comparison_note": "OUT OF SCOPE for a reuse kernel. Structured-output validity-vs-reasoning-tax (JSONSchemaBench, the answer-before-reasoning degradation) is a property of a constrained-generation engine fak does not have. fak claims nothing here; the related answershape consumer witness measures a different thing (degeneration), not schema conformance.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "tool-call-parse-robustness",
      "dim_id": "tool-call-parse-robustness",
      "axis": "Tool/function-call parsing coverage and robustness (per-model formats)",
      "category": "structured-output",
      "competitor": "vLLM tool-call parsers; SGLang; XGrammar-2 (grammar-backed tool calling)",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://docs.vllm.ai/en/latest/features/tool_calling/ (2025-12)",
      "fak_value": null,
      "unit": "parser coverage / malformed-call rate",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md Pre-flight ladder grammar rung [SHIPPED]: positional->named auto-repair (in-syscall TRANSFORM, no model turn) for arity-matched calls; unrepairable => Deny(MISROUTE); fail-open on unknown grammar (units 52-57). CLAIMS adjudicator: write-scoped codelint Deny(MALFORMED) on unparseable Go/JSON (#536). Gateway mints/validates calls. But there is NO per-model-family parser suite and NO measured coverage/robustness number.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP on the measured axis; mechanism is adjacent but narrow. fak has a SHIPPED boundary mechanism that hardens tool calls (arity repair + MISROUTE deny + MALFORMED deny on bad writes), which is robustness-shaped, but it is NOT the per-family format-parser coverage the SOTA bar measures (vLLM's hermes/llama3_json/mistral/pythonic/qwen3_coder parsers) and carries no coverage/malformed-rate number. So fak makes no coverage claim. Grammar-backed (constrained) tool calling — the robustness frontier — is the [STUB] decode-time mask fak has not built.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "sampler-coverage-correctness",
      "dim_id": "sampler-coverage-correctness",
      "axis": "Sampler coverage and correctness (temperature, top-p, top-k, min-p)",
      "category": "structured-output",
      "competitor": "vLLM; llama.cpp; SGLang; TensorRT-LLM",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://arxiv.org/abs/2407.01082 (2024-07)",
      "fak_value": null,
      "unit": "samplers supported (temp/top-p/top-k/min-p)",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No fak evidence: CLAIMS.md In-kernel model describes 'a real greedy Prefill+Step decode' and 'real greedy speculative decode is token-identical to plain greedy' — the decode path is GREEDY-ONLY. No temperature, top-p, top-k, or min-p sampler is mentioned in CLAIMS.md or BENCHMARK-AUTHORITY.md.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP. fak's in-kernel model decodes greedily only (argmax), which is what makes its losslessness proofs tractable, but it ships NO stochastic samplers (temperature/top-p/top-k/min-p). min-p is now native across vLLM/llama.cpp/SGLang/TensorRT-LLM; fak has none. A production serving engine needs samplers; fak makes no claim. (When fronting an external engine via fak guard / fak serve, sampling passes through to that engine untouched — fak's own engine has no sampler.)",
      "measured_on": "2026-06-23"
    }
  ]
}