imp AI技能包

Build from source

Tracks main rather than the latest release.

git clone https://github.com/kekzl/imp.git && cd imp
docker compose build imp-server
docker run --gpus all -v ./models:/models -v imp-cache:/home/imp/.cache/imp \
  -p 127.0.0.1:8080:8080 \
  imp:latest --model /models/your-model.gguf

Contributor workflow, build targets and the test lanes: CONTRIBUTING.md.

60-second quickstart

Everything runs in Docker; the host needs no CUDA toolkit. The worked example is Qwen3.8-27B, a 27B multimodal model quantized to NVFP4 so it fits one 5090 with room for a real context. The 60 seconds start once the weights are on disk; the two steps before that are a build and a download.

1. Build the image (~3.5 min). scripts/stage-model.sh runs the downloader inside the imp:test image this produces and exits 1 without it (#1682). docker compose build imp-server makes imp:latest, a different tag.

make build

2. Get the weights (19.2 GiB, download only, nothing to convert). The checkpoint is kekzle/Qwen3.8-27B-NVFP4-vllm, an imp-quantize --format vllm export; the same directory also loads in vLLM (verified on 0.27.1 and 0.28.0).

scripts/stage-model.sh kekzle/Qwen3.8-27B-NVFP4-vllm ~/models/Qwen3.8-27B-NVFP4-vllm

3. Serve. The cache volume is optional and pays for itself on the second start: it holds the transformed weights (Qwen3-14B-NVFP4 init 7.9 s → 2.1 s).

docker run --gpus all -v ~/models:/models -v imp-cache:/home/imp/.cache/imp \
  -p 127.0.0.1:8080:8080 ghcr.io/kekzl/imp:latest --model /models/Qwen3.8-27B-NVFP4-vllm

4. Ask. Open <http://localhost:8080>. The built-in UI streams the answer, draws one bar per token as it arrives, and shows the server's own counts for the run: prompt tokens, prefix-cache hits, reasoning tokens, context used. If the models directory holds more than one model, the header becomes a picker and choosing an unloaded entry swaps it in.

<p align="center"> <img src="docs/webui.png" width="900" alt="the built-in web UI after one answer: transcript on the left, one bar per token under it, run and usage readouts on the right"> </p>

_{Captured against the UI's dev mock (tools/imp-server/webui/dev/): the numbers in the picture are the mock's, not a measurement.}

Or from the shell:

curl -s http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen3.8-27B-NVFP4-vllm","messages":[{"role":"user","content":"Why is the sky blue?"}],"max_tokens":64}'

Expected: a JSON completion object whose choices[0].message.content explains Rayleigh scattering. The model id is the file or directory basename, GET /v1/models lists it, and the field is required. A GGUF file works the same way: swap the path.

The weights take 18.3 GiB on the card and answer at ~102 tok/s, leaving ~7 GiB for the KV cache on a 32 GB 5090.

[PROV: commit=f243179c date=2026-08-31 hw=RTX5090 model=Qwen3.8-27B-NVFP4-vllm quant=NVFP4 cuda=13.3 path=nvfp4-safetensors n=2 image=ghcr.io/kekzl/imp:latest cmd=imp-cli --model … --prompt … --max-tokens 128 --temperature 0 (102.8/101.9 tok/s)]