{
  "_about": "fak position rows for the 'models' group. dim_id links each to a _taxonomy.json dimension.",
  "rows": [
    {
      "id": "hardware-format-coverage",
      "dim_id": "hardware-format-coverage",
      "axis": "Hardware / format coverage and portability of quantized kernels",
      "category": "hardware-coverage",
      "competitor": "vLLM (FP8 on Hopper/Ada/Blackwell + MI300X via hipBLASLt/aiter); SGLang (FP8 MI300X); TensorRT-LLM (NVFP4 Blackwell); llama.cpp (broad CPU/GPU GGUF + FP4)",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://docs.vllm.ai/en/stable/features/quantization/fp8/ (2026-01)",
      "fak_value": null,
      "unit": "accelerated quant formats × hardware targets",
      "higher_is_better": true,
      "verdict": "trails",
      "status": "shipped",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "BENCHMARK-AUTHORITY.md + CLAIMS.md: fak runs f32 + Q8_0 on CPU (arm64/x86 SIMD), Vulkan (RX 7600), and CUDA (RTX 4070 sm_89, datacenter GPU); GGUF q4_k_m load proven (Qwen3.6-27B) but slow. NO FP8/INT8-W8A8/INT4-accelerated/NVFP4/MXFP4 kernel on any target",
      "apples_to_apples": false,
      "comparison_note": "A real, disclosed head-to-head where fak TRAILS on coverage breadth. fak's hardware portability is genuine (CPU SIMD + Vulkan + CUDA, with bit-exact cross-platform reproduction), but its FORMAT coverage is narrow: f32 plus 8-bit Q8_0, with GGUF k-quant load-only. The SOTA stacks accelerate FP8 across Hopper/Ada/Blackwell + MI300X, NVFP4 on Blackwell, and broad GGUF k-quants (llama.cpp). fak has no hardware-accelerated FP8/INT4/FP4 GEMM on ANY device, so on the format×hardware coverage axis it clearly trails — named, not buried.",
      "measured_on": "2026-06-23",
      "qualitative": true
    },
    {
      "id": "max-model-and-moe-coverage",
      "dim_id": "max-model-and-moe-coverage",
      "axis": "Maximum parameter count & frontier-MoE architecture coverage",
      "category": "model-coverage",
      "competitor": "llama.cpp (72B CPU / ~32B Metal; Qwen3-30B-A3B MoE @ 50 tok/s) · SGLang (multi-GPU TP, 27B+ quantized)",
      "competitor_class": "sota",
      "competitor_value": 72,
      "competitor_source": "HERO-BENCHMARK-2026-06-21.md ('Model-size ceiling on this box' fence); CLAIMS.md; llama.cpp model support (72B CPU / Qwen3-30B-A3B MoE)",
      "fak_value": 7,
      "unit": "B params (max on own engine)",
      "higher_is_better": true,
      "verdict": "trails",
      "status": "shipped",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "HERO-BENCHMARK-2026-06-21.md (honest fence) · CLAIMS.md",
      "apples_to_apples": false,
      "comparison_note": "fak's OWN generic GGUF-dequant engine ceilings at ~7B on 36 GB (dequant-to-f32 OOMs above; two arch families fail at load; f32 27B = 108 GB > 80 GB VRAM). fak's actual DeepSeek-class-MoE serving path is the in-kernel GLM-5.2 work (#1010, #1026): it runs the MoE/FFN experts + router on its own pure k_q8_gemm GPU kernel (cosine=1.0, commit 498a4ab) — but on a SINGLE device, with the DSA sparse-attention still host-side and no cross-node expert parallelism (see expert-parallelism-large-moe and attention-arch-kv-footprint). So for a frontier 671B MoE — the new MLPerf v6.0 / DeepSeek-R1 reference workload — fak is NOT in the race on its own engine and must FRONT llama.cpp/SGLang, exactly what the 27B GPU server run does (SGLang-serves + fak-adjudicates). A hard, disclosed capability ceiling that bounds where any fak engine 'win' can apply. apples_to_apples=false: fak's dequant/single-device engine vs llama.cpp quantized/CPU and SGLang multi-GPU TP are different configs by construction.",
      "measured_on": "2026-06-21"
    },
    {
      "id": "multimodal-vlm-coverage",
      "dim_id": "multimodal-vlm-coverage",
      "axis": "Multimodal / vision-language (VLM) model coverage",
      "category": "model-coverage",
      "competitor": "SGLang; vLLM",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://github.com/sgl-project/sglang (2026-01-01)",
      "fak_value": null,
      "unit": "VLM families served",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "none — no VLM in CLAIMS; in-kernel model is text-only SmolLM2/Qwen forward",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE for a text-tool-loop reuse kernel. fak's in-kernel model is a text-only forward pass (SmolLM2-135M / Qwen2.5 / GLM-5.2); there is NO vision encoder, no image-token path, no multimodal RadixAttention. fak's prefix-reuse mechanism is modality-agnostic in principle but no VLM is wired or measured. No fak number; verdict no-claim. (A REAL gap only if fak intended to serve VLMs on its own engine — it does not; it fronts an external engine for anything beyond ~7B text.)",
      "measured_on": "2026-06-23"
    },
    {
      "id": "long-context-capacity",
      "dim_id": "long-context-capacity",
      "axis": "Long-context serving (128K-1M tokens) & KV/prefix-cache reuse",
      "category": "model-coverage",
      "competitor": "SGLang (RadixAttention); vLLM (PagedAttention/prefix caching)",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://particula.tech/blog/sglang-vs-vllm-inference-engine-comparison (2026-01-01)",
      "fak_value": null,
      "unit": "% cache hit rate (FCFS 62.1% -> cache-aware), inside SGLang's published 50-99% band",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "experiments/radixattention/radixbench-qwen2.5-1.5b-q8-agents-fresh-20260619.json (workloads[0].cache_hit_rate=0.8667), commit 92896a4; BENCHMARK-AUTHORITY RadixAttention rows",
      "apples_to_apples": true,
      "comparison_note": "fak's 86.7% prefix-cache hit rate does NOT measure long-context capacity. fak's real long-context posture is its model-size ceiling (~7B on 36 GB) plus an arithmetic ultra-long-context WORK floor — not a served-capacity-vs-SOTA number. Honest gap.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "lora-multi-adapter-serving",
      "dim_id": "lora-multi-adapter-serving",
      "axis": "LoRA / multi-LoRA hot-swap serving",
      "category": "model-coverage",
      "competitor": "vLLM; SGLang; TensorRT-LLM",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://docs.vllm.ai/en/latest/features/lora/ (2025-01-01)",
      "fak_value": null,
      "unit": "concurrent LoRA adapters per base",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "none — no LoRA in CLAIMS",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE for the reuse/adjudication kernel. fak has no LoRA adapter loading, no multi-adapter batching, no hot-swap path. Its multi-model story is the polymodel residency Pool (host many full models, share prefill, decode one) — a DIFFERENT primitive (full models, not low-rank adapters) and itself off-mainline/policy-only (CLAIMS polymodel rows, FAK_POLYMODEL default off). No fak LoRA number; verdict no-claim.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "embeddings-rerankers-coverage",
      "dim_id": "embeddings-rerankers-coverage",
      "axis": "Embedding & reranker (non-generative) model coverage",
      "category": "model-coverage",
      "competitor": "vLLM; SGLang; TensorRT-LLM",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://docs.vllm.ai/en/latest/ (2026-01-01)",
      "fak_value": null,
      "unit": "embedding/reranker families served",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "none — no pooling/embedding endpoint in CLAIMS",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE for a generative-tool-loop reuse kernel. fak serves only causal-LM next-token decode; there is no pooling head, no embeddings endpoint, no cross-encoder reranker path. fak's OpenAI-compatible surface is /v1/chat/completions (CLAIMS Gateway), not /v1/embeddings. No fak number; verdict no-claim.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "hardware-backend-breadth",
      "dim_id": "hardware-backend-breadth",
      "axis": "Hardware backend breadth (NVIDIA, AMD, Intel, TPU, AWS, Apple, CPU)",
      "category": "hardware-coverage",
      "competitor": "vLLM; llama.cpp; SGLang; MLC-LLM",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://github.com/vllm-project/vllm (2026-01-01)",
      "fak_value": 4,
      "unit": "distinct compute backends shipped (CPU + NVIDIA CUDA + AMD Vulkan + Apple Metal), all single-device",
      "higher_is_better": true,
      "verdict": "trails",
      "status": "shipped",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS Engine: pure-Go CPU; '-tags vulkan' RX 7600; '-tags cuda' RTX 4070 / datacenter GPU; 'darwin && metal' M3 Pro — each numerically witnessed (cosine=1.0 / argmax-exact)",
      "apples_to_apples": true,
      "comparison_note": "REAL but narrow vs the SOTA breadth bar. fak ships four numerically-correct single-device backends: pure-Go CPU (x86 AVX/arm64) plus NVIDIA CUDA (RTX 4070 + 8-GPU datacenter server sm_80), AMD Vulkan (RX 7600), and Apple Metal (M3 Pro), each argmax-exact / cosine=1.0 vs cpu-ref. But vLLM (TPU, Gaudi 2/3, Inferentia/Trainium, Ascend, Spyre) and llama.cpp (HIP, SYCL, CANN, OpenCL, WebGPU via MLC) span far more silicon, and fak's GPU paths are mostly f32 and throughput-immature (Vulkan ~58x slower than llama.cpp CPU; CUDA reaches parity only on a small model that fits VRAM). fak trails on breadth and on per-backend maturity; disclosed, not hidden.",
      "measured_on": "2026-06-23",
      "qualitative": true
    },
    {
      "id": "served-model-capability-quality",
      "dim_id": "served-model-capability-quality",
      "axis": "Capability/quality of the served model (SWE-bench Verified, GPQA, AIME, LiveCodeBench)",
      "category": "model-coverage",
      "competitor": "Gemini 3.x (SWE-bench/GPQA); Claude Opus 4.5 + agent scaffold; GPT-5.x (AIME)",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://epoch.ai/benchmarks/swe-bench-verified (2025-11-24)",
      "fak_value": null,
      "unit": "% resolved (SWE-bench Verified) / benchmark accuracy",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "industry_scorecard.data.json swebench-resolve-rate row (no-claim/stub); SOTA-COMPARISON.md; CLAIMS: fak is a serving kernel, resolve rate set by the served model",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE by construction. fak is a serving KERNEL, not a model: SWE-bench Verified / GPQA / AIME / LiveCodeBench are set entirely by whatever model it serves, so fak claims NO resolve rate of its own. The frontier bars cited (SWE-bench Verified ~79-80.6%, GPQA Diamond ~93-94%, AIME up to 100%) are model-served-by-engine results. fak's claim is orthogonal: ~20-24x prefill-work reduction ON TOP of any model at the same quality, lower infra cost. Carried so the scorecard never quietly omits the headline a frontier-model comparison leads with. Honest no-claim.",
      "measured_on": "2026-06-23"
    }
  ]
}