llmfit AI技能包

Plan required hardware for a specific model configuration

llmfit plan "Qwen/Qwen3-4B-MLX-4bit" --context 8192 llmfit plan "Qwen/Qwen3-4B-MLX-4bit" --context 8192 --quant mlx-4bit llmfit plan "Qwen/Qwen3-4B-MLX-4bit" --context 8192 --target-tps 25 --json

Publish for real (requires a crates.io API token)

cargo login cargo publish

Before publishing, make sure:

- The version in `Cargo.toml` is correct (bump with each release).
- A `LICENSE` file exists in the repo root. Create one if missing:

Dependencies

---

Requirements

• Ollama must be installed and running (ollama serve or the Ollama desktop app)
• llmfit connects to http://localhost:11434 (Ollama's default API port)
• No configuration needed — if Ollama is running, llmfit detects it automatically

Install

Docker / Podman

docker run ghcr.io/alexsjones/llmfit

podman run ghcr.io/alexsjones/llmfit recommend --use-case coding | jq '.models[].name'

Docker Model Runner integration

llmfit integrates with Docker Model Runner, Docker Desktop's built-in model serving feature.

Requirements:

• Docker Desktop with Model Runner enabled
• Default endpoint: http://localhost:12434

How it works:

• llmfit queries GET /engines to list models available in Docker Model Runner
• models are matched to the HF database using Ollama-style tag mapping (Docker Model Runner uses ai/<tag> naming)
• pressing d in the TUI pulls via docker model pull

Remote Docker Model Runner instances

To connect to Docker Model Runner on a different host or port, set the DOCKER_MODEL_RUNNER_HOST environment variable:

DOCKER_MODEL_RUNNER_HOST="http://192.168.1.100:12434" llmfit

Install the skill

```sh

Usage

Recommendations filtered by use case

llmfit recommend --json --use-case coding --limit 3

Advanced Configuration (`A`)

Press A to open the Advanced Configuration popup. This panel lets you tune the parameters behind TPS estimation, run mode penalties, and composite scoring — addressing issue #449 where tok/s was overestimated for certain models (e.g., Qwen3 30B).

All changes are applied immediately and the model table is recalculated. Close with Esc to accept or Ctrl-R to reset to defaults.

Via environment variable (recommended)

export LOCALMAXXING_API_KEY="bhk_your_key_here" llmfit

Or via CLI flag

llmfit --api-key "bhk_your_key_here" ```

Override endpoint URL

llmfit bench --provider ollama --url http://my-server:11434 llama3.2

Override vLLM endpoint

llmfit bench --provider vllm --url http://localhost:8000

CLI mode

Use --cli or any subcommand to get classic table output:

```sh

Run as a node-level REST API (for cluster schedulers / aggregators)

llmfit serve --host 0.0.0.0 --port 8787 ```

REST API (`llmfit serve`)

llmfit serve starts an HTTP API that exposes the same fit/scoring data used by TUI/CLI, including filtering and top-model selection for a node.

```sh

Key scheduling endpoint: top runnable models for this node

curl "http://localhost:8787/api/v1/models/top?limit=5&min_fit=good&use_case=coding"

spawn server automatically and run endpoint/schema/filter assertions

python3 scripts/test_api.py --spawn

Works with all TUI and CLI commands

OLLAMA_HOST="http://192.168.1.100:11434" llmfit --cli OLLAMA_HOST="http://192.168.1.100:11434" llmfit fit --perfect -n 5 ```

This is useful for: - Running llmfit on one machine while Ollama serves from another (e.g., GPU server + laptop client) - Connecting to Ollama running in Docker containers with custom ports - Using Ollama behind reverse proxies or load balancers

API authentication

If your LM Studio instance has Require API Key enabled (required for MCP server access), set the LMSTUDIO_API_KEY environment variable to provide a Bearer token with all requests:

export LMSTUDIO_API_KEY="your-api-key-here"
llmfit

Runtime provider integration

llmfit supports multiple local runtime providers:

• Ollama (daemon/API based pulls)
• llama.cpp (direct GGUF downloads from Hugging Face + local cache detection)
• MLX (Apple Silicon / mlx-community model cache + optional server) — MLX downloads map to mlx-community/* repos on HuggingFace, not the original model publisher
• Docker Model Runner (Docker Desktop's built-in model serving)
• LM Studio (local model server with REST API for model management + downloads)

When more than one compatible provider is available for a model, pressing d in the TUI opens a provider picker modal.

Ollama integration

llmfit integrates with Ollama to detect which models you already have installed and to download new ones directly from the TUI.

llama.cpp integration

llmfit integrates with llama.cpp as a runtime/download provider in both TUI and CLI.

Requirements:

• llama-cli or llama-server available in PATH (for runtime detection)
• network access to Hugging Face for GGUF downloads

How it works:

• llmfit maps HF models to known GGUF repos (with heuristic fallbacks)
• downloads GGUF files into the local llama.cpp model cache
• marks models installed when matching GGUF files are present locally

Environment variables

If llama.cpp is installed in a non-standard location, set LLAMA_CPP_PATH so llmfit can find it without requiring it in your PATH.

LM Studio integration

llmfit integrates with LM Studio as a local model server with built-in model download capabilities.

Requirements:

• LM Studio must be running with its local server enabled
• Default endpoint: http://127.0.0.1:1234

How it works:

• llmfit queries GET /v1/models to list models available in LM Studio
• pressing d in the TUI triggers a download via POST /api/v1/models/download
• download progress is tracked by polling GET /api/v1/models/download-status
• LM Studio accepts HuggingFace model names directly, so no name mapping is needed

OpenClaw integration

llmfit ships as an OpenClaw skill that lets the agent recommend hardware-appropriate local models and auto-configure Ollama/vLLM/LM Studio providers.

Alternatives

If you're looking for a different approach, check out llm-checker -- a Node.js CLI tool with Ollama integration that can pull and benchmark models directly. It takes a more hands-on approach by actually running models on your hardware via Ollama, rather than estimating from specs. Good if you already have Ollama installed and want to test real-world performance. Note that it doesn't support MoE (Mixture-of-Experts) architectures -- all models are treated as dense, so memory estimates for models like Mixtral or DeepSeek-V3 will reflect total parameter count rather than the smaller active subset.

---