骆驼

Install

Two ways to run Camelid — both use the same engine and the same models. Pick what fits:

The desktop app simply wraps the same engine in a native window — same models, same support gate, same GPU acceleration. If you just want to chat, get the desktop app. If you want to share it or use the API, get the engine.

Build from source on Windows (x86_64, MSVC)

Windows x86_64-pc-windows-msvc is a tracked platform (see COMPATIBILITY.md → Platform support). Most users should grab the prebuilt signed Windows download in Install above (GPU acceleration included); build from source only if you want to modify Camelid. Prerequisites: the MSVC toolchain (Visual Studio Build Tools with the C++ workload — not MinGW), Rust via rustup with the x86_64-pc-windows-msvc host, and Node.js for the embedded web UI. Then, in PowerShell:

cd frontend; npm ci; npm run build; cd ..   # bundles the web UI
cargo build --release                       # embeds it into the binary
.\target\release\camelid.exe pull tinyllama # the baseline supported row
.\target\release\camelid.exe serve --model models\tinyllama-1.1b-chat-v1.0.Q8_0.gguf

The server behaves exactly as on the other platforms (listens on 127.0.0.1:8181, same OpenAI-style API + web UI). The TinyLlama 1.1B Chat Q8_0 baseline gate is verified on Windows with the same parity evidence as macOS/Ubuntu.

GPU (NVIDIA/CUDA) on Windows. cargo build --release --features cuda adds a CUDA backend with a GPU-resident decode engine (weights uploaded once, single-shot GPU prefill, on-device greedy/temperature sampling) implemented from scratch in NVRTC kernels — no vendored llama.cpp. It auto-engages when a CUDA device is present. The dense Qwen3 Q8_0 rows (0.6B / 1.7B / 4B / 8B Instruct, thinking-disabled ChatML) are validated on it: GPU decode + single-shot prefill are token-AND-text-identical to the camelid cpu_reference (transitively llama.cpp 9632) at 1/5/50 generated tokens, greedy. Gemma 4 E4B-It Q8_0 also runs on this CUDA lane (enabled with CAMELID_GEMMA4_CUDA), greedy-parity with the CPU Gemma4Runtime oracle via the in-tree gate (gemma4_cuda_matches_cpu_greedy); it has no committed evidence bundle yet, so it stays experimental beyond the recorded GPU. /api/capabilities reports the live path (selected_backend=cuda_resident_q8_runtime, cuda_resident_active=true). Validated on an RTX 3060 Laptop (6 GB), driver 576.83, CUDA 12.9; 0.6B/1.7B/4B are fully VRAM-resident and 8B runs via automatic VRAM+host-RAM layer offload. Results are GPU/driver/CUDA-version specific (f32 reduction order is GPU-specific), so the lane stays experimental beyond the recorded GPU; the CPU path remains the default and correctness reference. See COMPATIBILITY.md → Windows CUDA and the qa/evidence-bundles/qwen3-*-windows-cuda-resident-parity-* bundles. Building the feature needs the CUDA Toolkit (12.x; libraries loaded at runtime); running it needs an NVIDIA GPU + driver.

Get a model. Camelid validates specific Q8_0 rows (most GGUFs on the web are other quantizations and fail closed), so pull fetches a known-good one into ./models:

./target/release/camelid pull              # list the supported models
./target/release/camelid pull llama32_3b   # download Llama 3.2 3B Instruct Q8_0

Serve it (pull prints the exact command to run; the model is in ./models):

./target/release/camelid serve \
  --model models/Llama-3.2-3B-Instruct-Q8_0.gguf \
  --threads 4

The server listens on 127.0.0.1:8181 and opens the chat UI in your browser automatically (pass --no-open to disable). The same address serves the OpenAI-style API. List the loaded model (its id comes from the GGUF metadata):

curl -s http://127.0.0.1:8181/v1/models

Chat (replace the model id with the one returned above; add "stream": true for SSE):

curl -s http://127.0.0.1:8181/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Llama 3.2 3B Instruct",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}],
    "max_tokens": 64,
    "temperature": 0
  }'

The web frontend is served by the binary itself at the same address — no extra step. For hot-reloading frontend development, run the Vite dev server separately (it proxies to a running camelid serve):

cd frontend && npm ci && npm run dev

---

Quickstart

Already have a binary from Install? Skip to "Get a model" below. To build from source instead — the web UI is compiled into the binary, so build the frontend first and it gets embedded (one binary, no separate Node process at runtime):

(cd frontend && npm ci && npm run build)   # bundles the web UI
cargo build --release                       # embeds it into the binary

Supported models

Support is per exact model row (a specific GGUF at a specific quantization), each backed by committed evidence. Anything not listed fails closed.

Fails closed (by design): Mixtral-8x7B v0.1 (validation-in-progress, one-token runtime only); other Qwen3 sizes (14B/32B), base variants, Qwen3-MoE (A3B), and full-trace Qwen3 thinking-mode token-parity (thinking is available opt-in with leading-trace parity); Gemma 4 26B-A4B Q8_0 (26.9 GB) and 31B (over the 2×16 GB envelope); Gemma 4 MTP/drafter rows; DiffusionGemma 26B-A4B on the autoregressive engine (a discrete block-diffusion model cannot run an AR forward — the AR engine fails closed and redirects to the dedicated diffusion-gemma-chat lane, which is supported; see below); multimodal input; and all other quantizations in v0.1.