Zinc本地推理引擎

Prerequisites

Important: On Linux with RDNA4, newer glslc releases can cause a large regression. Use the system package version.

Build ZINC

```bash git clone https://github.com/zolotukhin/zinc.git cd zinc

Build the CLI and server

Linux: shaders are compiled automatically

zig build -Doptimize=ReleaseFast ```

The binary is placed in zig-out/bin/zinc. Compiled SPIR-V shaders go to zig-out/share/zinc/shaders/. Use ReleaseFast for any performance measurement or server deployment. Plain zig build is not a fair throughput baseline.

Quick Start

Use the API

ZINC exposes an OpenAI-compatible API at /v1.

For the actual request examples and SDK usage, use the website docs instead of the README:

• Running ZINC for CLI, server mode, and first-run examples
• Serving HTTP API for curl, OpenAI SDK examples, endpoint behavior, and response shapes

The built-in chat UI is served at /, the API is under /v1, and the health endpoint is /health.

Supported Models

The list below matches the current managed model catalog, not a broader wishlist.

• Qwen3.6 35B-A3B UD Q4_K_XL — supported on AMD RDNA4 32 GB and Apple Silicon
• Qwen3.6 27B Dense Q4_K_M — experimental on AMD RDNA4 32 GB and Apple Silicon
• OpenAI GPT-OSS 20B Q4_K_M — supported on Apple Silicon
• Qwen3 8B Q4_K_M — supported on AMD RDNA4 32 GB and Apple Silicon
• Gemma 4 31B Q4_K_M — supported on AMD RDNA4 32 GB and Apple Silicon
• Gemma 4 12B (26B-A4B MoE) Q4_K_M — experimental on AMD RDNA4 32 GB and Apple Silicon

• Use zinc model list --json for machine-readable model metadata
• Current throughput and latency numbers live on the public benchmarks page: zolotukhin.ai/zinc/benchmarks

Quantization formats: Q4_K, Q5_K, Q6_K, Q8_0, Q5_0, MXFP4, F16, F32

Where we stand vs llama.cpp

• Ahead of llama.cpp: Qwen 3 8B prefill on RDNA4 (1.36x), Qwen 3.6 35B-A3B decode on RDNA4 (1.12x), Gemma 4 31B dense decode on RDNA4 (1.38x).
• Within striking distance (>=90% of llama.cpp): Gemma 4 26B A4B decode on RDNA4 (97%).
• Active gap: Qwen 3.6 35B-A3B prefill on RDNA4 sits at ~48% of llama.cpp because the entire batched prefill path is gated off for any model with n_experts > 0 or ssm_d_inner > 0. The wire-up that closes this is documented in the cycle-50 field report.
• In flight: Metal prefill is uniformly bottlenecked (20-30 tok/s) because the per-token Metal path doesn't amortize weight reads across prompt tokens. The Gemma 4 31B decode floor at 5.16 tok/s is the active optimization target.

For local benchmark commands, harnesses, and methodology, see:

• Development Guide
• Running ZINC