{
  "_about": "fak position rows for the 'operability' group. dim_id links each to a _taxonomy.json dimension.",
  "rows": [
    {
      "id": "fairness-isolation-multitenant",
      "dim_id": "fairness-isolation-multitenant",
      "axis": "Fairness, priority, and tenant isolation in batch formation / scheduling",
      "category": "operability",
      "competitor": "FairBatching (fairness-aware batch formation); SGLang (FCFS + page-eviction preemption); hybrid real-time/best-effort schedulers (e.g. arXiv 2504.09590)",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://arxiv.org/pdf/2510.14392 (2025-10)",
      "fak_value": null,
      "unit": "per-request unfairness / priority-preemption",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No fairness / priority-class / tenant-isolation batch-formation number in BENCHMARK-AUTHORITY.md or CLAIMS.md. fak ships per-agent KV ownership and causal cache invalidation (BENCHMARK-AUTHORITY.md causal-invalidation witness, max|delta|=0) but no measured head-of-line-blocking or priority-preemption behavior.",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE as a measured serving-fairness number, though adjacent to fak's trust plane: fak provides per-agent KV isolation and targeted causal eviction (witnessed bit-exact), but it has no FairBatching-style per-request unfairness bound, no priority classes with preemption, and no measured head-of-line-blocking behavior in batch formation. The SOTA bar here is itself qualitative; fak still has no number, so no claim.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "ttft-degradation-under-load",
      "dim_id": "ttft-degradation-under-load",
      "axis": "Latency degradation under concurrency / load (saturation behavior)",
      "category": "operability",
      "competitor": "vLLM benchmark_serving (request-rate sweep); NVIDIA GenAI-Perf (concurrency sweep)",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://tianpan.co/blog/2026-03-10-llm-latency-decomposition-ttft-vs-throughput (2026-03)",
      "fak_value": null,
      "unit": "RPS at which P99 TTFT crosses SLO",
      "higher_is_better": false,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "The GPU server concurrency sweep (conc 1->128, QWEN36-27B-GPU-SERVER-RESULTS.md) varies LOAD but records completion-tokens/sec, not a TTFT/queueing curve: there is no committed P99-TTFT-vs-RPS saturation sweep showing the rate at which the SLO is crossed.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP fak should measure. fak has the harness to vary concurrency (it ran 1->128 on the GPU server) but reports the throughput envelope, not the saturation diagnostic (P99 TTFT climbing while P50 stays flat as the queue builds). Since the gateway proxy adds queueing of its own, the TTFT-under-load curve is exactly what an honest operability claim needs; not yet measured, so no-claim.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "cold-start-scaling-latency",
      "dim_id": "cold-start-scaling-latency",
      "axis": "Cold-start / autoscaling latency (scale-to-zero & scale-out)",
      "category": "operability",
      "competitor": "RunPod FlashBoot; ServerlessLLM; HydraServe; PipeBoost",
      "competitor_class": "sota",
      "competitor_value": 250,
      "competitor_source": "https://regolo.ai/scale-to-zero-cold-start-latency-why-serverless-gpu-breaks-real-time-ai-and-how-to-fix-it/ (2025-01)",
      "fak_value": null,
      "unit": "ms/s cold-start to first token",
      "higher_is_better": false,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No cold-start, scale-to-zero, or autoscaling latency is measured in CLAIMS.md / BENCHMARK-AUTHORITY.md. fak has a model-residency/budget policy plane (internal/residency, internal/cachemeta) but it is off-mainline (FAK_POLYMODEL-gated) and moves no weight bytes - no checkpoint-load or startup-to-first-token timing is committed.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP, partially out of scope. Cold-start/scale-to-zero latency (RunPod FlashBoot sub-250ms; ServerlessLLM 6-8x faster load) is a serverless-platform metric for the weight-loading path fak's own engine does not yet exercise at scale (its largest faithfully-served model is ~7B, it fronts external engines above that). fak has a residency/page-out POLICY but no measured load-latency number; no-claim.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "persistent-cross-instance-cache-coherence",
      "dim_id": "persistent-cross-instance-cache-coherence",
      "axis": "Cross-instance / persistent KV cache sharing and coherence",
      "category": "operability",
      "competitor": "LMCache; Mooncake Store",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://arxiv.org/html/2510.09665v2 (2025-10)",
      "fak_value": null,
      "unit": "cross-instance KV reuse",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "in-flight",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS lines 66-71: recall persists a finished session as a durable, integrity-checked CORE IMAGE reloadable in a FRESH process; BENCHMARK-AUTHORITY causal-invalidation row proves a coherence-bus broadcast + targeted causal eviction (max|Δ|=0). But this is cross-PROCESS session reload + invalidation coherence, not a live cross-INSTANCE shared KV POOL like LMCache/Mooncake Store.",
      "apples_to_apples": false,
      "comparison_note": "REAL GAP, with adjacent mechanism shipped. fak has durable cross-process session core images and a coherence bus that broadcasts cache refutations with byte-exact causal eviction, but NOT a global deduplicated KV pool shared across live engine instances (LMCache cross-instance reuse / Mooncake Store). The pieces that point toward it (recall, the coherence bus, the not-yet-witness-keyed tier-2) are real; the cross-instance live-sharing claim is unbuilt, so no parity is asserted.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "fault-tolerance-resilience",
      "dim_id": "fault-tolerance-resilience",
      "axis": "Multi-node fault tolerance & failure recovery (resilient serving)",
      "category": "operability",
      "competitor": "FailSafe (research); KevlarFlow (research); ReviveMoE (research); NVIDIA Dynamo",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://arxiv.org/html/2601.22438v1 (2026-01-01)",
      "fak_value": null,
      "unit": "mean-time-to-recovery / sustained throughput under GPU loss",
      "higher_is_better": false,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "none — no multi-node failure-recovery number in CLAIMS; causal-invalidation (commit 0fc39aa) is an integrity/coherence witness, not failure recovery",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE for a single-box reuse kernel. fak has no multi-node serving topology, hence no node-failure rerouting, no background KV replication, no decoupled parallel re-init. Its nearest adjacent witness — causal invalidation-on-external-write (vdso.Revoke, commit 0fc39aa, max|Delta|=0) — is an INTEGRITY/coherence property (evict cache reads a write made stale), explicitly 'a containment/coherence witness, not a throughput number', NOT failure recovery. The research bar (KevlarFlow/FailSafe/ReviveMoE) is multi-GPU resilient serving fak does not attempt. No fak number; verdict no-claim.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "openai-compatible-api-and-structured-output",
      "dim_id": "openai-compatible-api-and-structured-output",
      "axis": "OpenAI-compatible API surface + reliable structured output / tool-calling",
      "category": "operability",
      "competitor": "vLLM (XGrammar/guided decoding); SGLang; LiteLLM (gateway)",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://blog.squeezebits.com/guided-decoding-performance-vllm-sglang (2025-01-01)",
      "fak_value": null,
      "unit": "OpenAI-compat surface present (capability), structured-output rung depth",
      "higher_is_better": true,
      "verdict": "parity",
      "status": "shipped",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md Gateway [SHIPPED] (/v1/chat/completions, /v1/models, /healthz, MCP); Engine [SHIPPED] OpenAI-compatible client base_url-swappable; Pre-flight grammar rung [SHIPPED] positional->named repair; decode-time logit-mask [STUB]",
      "apples_to_apples": false,
      "comparison_note": "Parity on the INTEGRATION CONTRACT, not on guided-decoding conformance. fak ships the de facto OpenAI-compatible surface (/v1/chat/completions adjudication proxy, /v1/models, MCP over stdio/HTTP, base_url-swappable client) so a non-Go agent routes through the same boundary, and a grammar rung does in-syscall positional->named tool-call repair. But the STRONGEST structured-output form, decode-time logit-mask / grammar-constrained generation (XGrammar-style never-emit-malformed, the >96-98.2% JSON-conformance lever vLLM/SGLang cite) is explicitly [STUB] because it requires owning the decode loop. apples_to_apples=false: no committed JSON-schema conformance % to compare, so this is a capability-presence parity on the API surface, a gap on guaranteed schema decoding.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "observability-metrics-tracing",
      "dim_id": "observability-metrics-tracing",
      "axis": "Production observability: Prometheus metrics (TTFT/TPOT/queue) + OpenTelemetry tracing",
      "category": "operability",
      "competitor": "vLLM; SGLang; NVIDIA Dynamo",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://docs.vllm.ai/en/latest/design/metrics/ (2025-01-01)",
      "fak_value": null,
      "unit": "metric-set coverage vs the vLLM TTFT/TPOT/queue/cache-hit + OTel contract",
      "higher_is_better": true,
      "verdict": "trails",
      "status": "shipped",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "CLAIMS.md Gateway [SHIPPED] /metrics + Server.AdjudicationSummary + TraceID threaded end-to-end; Engine [SIMULATED] metrics-service scrape adapter (unit 43); no committed TTFT/TPOT/queue-depth/cache-hit Prometheus series",
      "apples_to_apples": true,
      "comparison_note": "fak exposes ADJUDICATION observability (a /metrics endpoint backing the exit AdjudicationSummary operation counters, an end-to-end TraceID the IFC ledger / plan-CFI key on) but NOT the serving-systems observability contract it is judged against: there is no committed per-request TTFT (vllm:time_to_first_token_seconds), TPOT/inter-token, queue depth, KV-cache usage or prefix-cache-hit-rate Prometheus series, and the metrics-service scrape adapter is [SIMULATED] telemetry. OpenTelemetry distributed traces per the vLLM v1 spec are not shipped. fak's tracing is adjudication-plane, the de facto serving metric-set is the gap. Honest trails.",
      "measured_on": "2026-06-23",
      "qualitative": true
    },
    {
      "id": "kubernetes-autoscaling-scale-to-zero",
      "dim_id": "kubernetes-autoscaling-scale-to-zero",
      "axis": "Kubernetes-native deployment + LLM-aware autoscaling (metric-driven, scale-to-zero)",
      "category": "operability",
      "competitor": "KServe + KEDA; Ray Serve; llm-d / vLLM on Kubernetes",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://developers.redhat.com/articles/2025/09/23/how-set-kserve-autoscaling-vllm-keda (2025-09-23)",
      "fak_value": null,
      "unit": "LLM-metric-driven autoscaling / scale-to-zero capability",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No KServe/KEDA/Ray/K8s deployment claim in CLAIMS.md or BENCHMARK-AUTHORITY.md; fak ships one statically-linked Go binary + fak guard / fak serve",
      "apples_to_apples": true,
      "comparison_note": "OUT OF SCOPE for a single-process reuse kernel. fak is one statically-linked Go binary (fak serve / fak guard) with no Kubernetes operator, no custom-LLM-metric HPA (vllm:num_requests_running), no KServe/KEDA/Ray integration and no scale-to-zero. The 2025-2026 vLLM+KServe/llm-d+Ray+Kueue+KEDA stack has no fak counterpart and fak does not claim one. Honest no-claim; would only become a gap if fak grew into a multi-replica serving platform.",
      "measured_on": "2026-06-23"
    },
    {
      "id": "multi-tenancy-fairness-slo",
      "dim_id": "multi-tenancy-fairness-slo",
      "axis": "Multi-tenant fairness, per-tenant quotas, and SLO attainment under contention",
      "category": "operability",
      "competitor": "VTC (Sheng et al.); LiteLLM (virtual keys/budgets); Equinox / FairBatching",
      "competitor_class": "sota",
      "competitor_value": null,
      "competitor_source": "https://arxiv.org/abs/2401.00588 (2024-01-01)",
      "fak_value": null,
      "unit": "per-tenant fairness / SLO attainment under contention",
      "higher_is_better": true,
      "verdict": "no-claim",
      "status": "stub",
      "fak_commit": "",
      "fak_artifact": "",
      "fak_doc": "No VTC-style fair-scheduler or per-tenant quota/budget claim in CLAIMS.md; lease-disjointness steward + per-agent KV ownership are isolation, not provable batch fairness",
      "apples_to_apples": true,
      "comparison_note": "REAL GAP for the multi-agent regime fak targets. fak has per-agent KV ownership and a lease-disjointness steward (isolation primitives) and an agent-scoped tainted Ref per wire client, but NO work-conserving fair scheduler (VTC-style cumulative per-client token tracking), no per-tenant token budgets / spend caps (the LiteLLM virtual-key axis), and no measured SLO-attainment-under-contention number. Since fak's product is fleets of agents, provable fairness is a gap it arguably SHOULD measure rather than out-of-scope. No fak number exists; honest no-claim.",
      "measured_on": "2026-06-23"
    }
  ]
}