AI Skill Hub 强烈推荐：TurboLLM 是一款优质的AI工具。AI 综合评分 8.0 分，在同类工具中表现稳健。如果你正在寻找可靠的AI工具解决方案，这是一个值得深入了解的选择。
本地LLM引擎自动优化工具，支持GPU加速
TurboLLM 是一款基于 TypeScript 开发的开源工具，专注于 AI、GPU、TypeScript 等核心功能。作为 GitHub 开源项目，它拥有活跃的社区支持和持续的版本迭代，代码完全透明可审计，支持本地部署以保护数据隐私。无论是个人使用还是集成到企业工作流，都能提供稳定可靠的解决方案。
本地LLM引擎自动优化工具，支持GPU加速
TurboLLM 是一款基于 TypeScript 开发的开源工具，专注于 AI、GPU、TypeScript 等核心功能。作为 GitHub 开源项目，它拥有活跃的社区支持和持续的版本迭代，代码完全透明可审计，支持本地部署以保护数据隐私。无论是个人使用还是集成到企业工作流，都能提供稳定可靠的解决方案。
# 方式一：npm 全局安装 npm install -g turbollm # 方式二：npx 直接运行（无需安装） npx turbollm --help # 方式三：项目依赖安装 npm install turbollm # 方式四：从源码运行 git clone https://github.com/mohitsoni48/TurboLLM cd TurboLLM npm install npm start
# 命令行使用
turbollm --help
# 基本用法
turbollm [options] <input>
# Node.js 代码中使用
const turbollm = require('turbollm');
const result = await turbollm.run(options);
console.log(result);
# turbollm 配置说明 # 查看配置选项 turbollm --config-example > config.yml # 常见配置项 # output_dir: ./output # log_level: info # workers: 4 # 环境变量（覆盖配置文件） export TURBOLLM_CONFIG="/path/to/config.yml"
<p align="center"> <img src="https://raw.githubusercontent.com/mohitsoni48/TurboLLM/main/turbollm/web/public/brand/turbollm-icon-512.jpeg?v=2" width="92" height="92" alt="TurboLLM" /> </p>
<p align="center"> <strong>Run <em>any</em> local LLM engine, auto-tuned to your GPU — with a polished web UI and an OpenAI/Anthropic-compatible API.</strong><br/> Bring your own llama.cpp fork. No compiling. No Electron. No Python. Point Claude Code at your own machine in one command — fully offline. </p>
<p align="center"> <a href="https://www.npmjs.com/package/turbollm"><img src="https://img.shields.io/npm/v/turbollm.svg?color=e2552e" alt="npm version" /></a> <a href="https://www.npmjs.com/package/turbollm"><img src="https://img.shields.io/npm/dm/turbollm.svg?color=e2552e" alt="npm downloads" /></a> <img src="https://img.shields.io/badge/node-%E2%89%A522-3c873a.svg" alt="node >= 22" /> <img src="https://img.shields.io/badge/license-FSL--1.1--ALv2-blue.svg" alt="license" /> <img src="https://img.shields.io/badge/platform-Windows%20%C2%B7%20macOS%20%C2%B7%20Linux-555.svg" alt="platforms" /> <a href="https://ko-fi.com/mohitsoni"><img src="https://img.shields.io/badge/Ko--fi-support%20us-FF5E5B?logo=kofi&logoColor=white" alt="Ko-fi" /></a> <a href="https://github.com/sponsors/mohitsoni48"><img src="https://img.shields.io/badge/GitHub%20Sponsors-support%20us-EA4AAA?logo=githubsponsors&logoColor=white" alt="GitHub Sponsors" /></a> <a href="https://discord.gg/v6kRbV7nC"><img src="https://img.shields.io/badge/Discord-join%20chat-5865F2?logo=discord&logoColor=white" alt="Discord" /></a> </p>
npx turbollmThat one command starts a local daemon, opens a browser UI, and serves your models over an API any tool can talk to. TurboLLM is the performance & bleeding-edge layer for local LLMs — built for people who today hand-compile forks and hunt forums for the right flags.
<p align="center"> <img src="https://raw.githubusercontent.com/mohitsoni48/TurboLLM/main/assets/how-it-works.svg?v=2" width="860" alt="How TurboLLM works: clients -> one lightweight daemon -> any engine on your GPU" /> </p>
---
The headline — running any engine, including community forks — has its own section below. Everything else is grouped here; each summary is the gist, expand for the detail:
<details> <summary><strong>📦 Models — bring your own, or browse Hugging Face</strong></summary>
<br/>
- Use the folders you already have. Point TurboLLM at any directory of GGUFs — your existing LM Studio / Ollama / manual downloads — no re-downloading. It parses GGUF metadata (arch, params, quant, context, vision) for every file. - Browse & download from Hugging Face, in-app: search, see the file tree, pick a quant, and download with resume + SHA-256 verification. Gated models (Llama, Gemma) work via your own HF token, which never leaves your machine. - Import from any URL — not just Hugging Face. Paste a direct .gguf link (model-author sites, mirrors, private servers); it disk-space-checks and downloads through the same manager. - Quant recommendation per GPU and a VRAM-fit verdict so you pick a quant that actually fits before you commit. - Primary download folder, real-time measured t/s per model, and delete-from-disk.
</details>
<details> <summary><strong>⚡ Auto-tuning & performance</strong></summary>
<br/>
- Auto-benchmark on load derives fast defaults for your exact GPU. - Recommended sampling from the model card — auto-tune reads the model's Hugging Face card (falling back to the original model behind a requant) and prefills the author's recommended temperature / top_k / top_p / min_p. No recommendation → your sampling is left untouched. - Real measured tokens/sec in the model list — live while generating, last-session when idle (never a synthetic estimate). - Full load-parameter UI, a superset of what other tools expose: context length, GPU offload (-ngl), MoE CPU-offload (--n-cpu-moe), parallel slots, KV-cache quant type (incl. low-bit on supporting forks), CPU threads, flash attention, and speculative decoding (NextN / MTP / draft). - Fast by default: flash attention on, NextN self-speculative decoding on for models that carry a draft head, threads auto — safely gated to what your engine actually accepts. - Multi-GPU, per model — split a model across cards (layer/row split + main-GPU pick on llama.cpp, tensor-parallel on vLLM). Defaults are no-ops, so single-GPU rigs are untouched. - Saved per-model profiles — tune once, and it loads that way every time.
</details>
<details> <summary><strong>💬 Chat & agentic tools — a genuinely good UI, not an afterthought</strong></summary>
<br/>
- Streaming with a stop button, live tokens/sec, prompt-processing % and prefill t/s, time-to-first-token, total time, exact token counts, and a context-usage meter (filled / max) on every reply. - Thinking control — toggle reasoning off for a direct answer, or leave it on with collapsible, timed "thought for N s" blocks. - Markdown + syntax-highlighted code with one-click copy — plus inline Unicode charts the model draws when a comparison, trend, or hierarchy is genuinely worth a visual. - Live artifacts — html, svg, and mermaid replies render as sandboxed, offline previews shown as an image, with one-click export to PNG / JPEG / SVG / animated GIF / HTML. - Personas — pick a style (Default · Designer · Concise · Detailed · Blunt · Formal · Tutor · Creative · Research) per conversation, no prompt-wrangling required. The Designer persona produces polished, self-contained, previewable designs by default. - Edit, regenerate, delete, copy any message; persistent, searchable conversations with rename, delete, and auto-generated titles. - Per-chat system prompt and per-chat sampling overrides — temperature, top-p/k, min-p, repeat/presence/frequency penalties, and stop strings. - Image input for vision models, and TurboLLM Expert — a built-in assistant that knows the app and your hardware for onboarding and troubleshooting without leaving the UI. - Agentic tools — built-in web_search (Tavily), fetch_url, and sandboxed run_code, plus MCP server support (stdio / SSE) so any MCP server's tools appear in every chat. A Research persona forces multi-step web search and cites sources inline.
</details>
<details> <summary><strong>🤖 Background agents — long-running tasks that don't tie up your chat</strong></summary>
<br/>
- Launch an agent and walk away. The Agents screen runs tasks in the daemon, separate from the chat tab — describe the task, pick which tools it may use (web search / fetch URL / run code), and let it work. - Live, reconnectable progress. Watch the run stream in real time; navigate away or reload and the view reconnects to the in-progress output. Runs queue behind any active run and persist across restarts. - Cancel anytime, and review completed runs (messages + the tool calls they made) later.
</details>
<details> <summary><strong>🔌 APIs & integrations — OpenAI + Anthropic, plus a model-loading gateway</strong></summary>
<br/>
With a model loaded, TurboLLM serves two compatible APIs on the same port:
```bash
No other local-LLM app lets you run whatever inference engine you want. TurboLLM treats the engine as a swappable component.
Add a custom engine (Engines screen → Add engine):
1. Compile or download any llama-server-compatible binary — stock llama.cpp, a community fork, or your own build. 2. Point TurboLLM at the folder — it scans for the llama-server binary, runs a capability probe, and learns exactly which flags and features that build supports. (Optional: paste the source repo URL so TurboLLM flags when a newer build ships.) 3. Activate it. The load-parameter UI adapts to that engine — features the build doesn't support are hidden; ones it adds (e.g. low-bit KV cache, NextN) light up.
No prebuilt for your OS? The build-from-source guide checks your toolchain (git / CMake / CUDA / MSVC), hands you the exact build commands, then drops you into the folder scan above.
Auto-provisioned default. Don't want to fetch anything? On first run TurboLLM downloads the right upstream prebuilt for your GPU automatically — and a backend picker lets you switch between CUDA / ROCm / Metal / SYCL / Vulkan / CPU at any time (it downloads the variant you choose, LM Studio-style).
Engine types. llama.cpp / GGUF, KoboldCpp and llamafile (GGUF, every OS), MLX (macOS), and vLLM (Linux + NVIDIA) are all first-class engine kinds — install from the curated catalog, pick the right one per model, and switch from a single dropdown.
Fully supervised. Every engine runs under a real state machine: health-gated readiness, graceful stop, an idle auto-stop watchdog, and live logs + clear error surfacing in the UI when something fails to load.
Why it matters: fork-exclusive features — speculative decoding (NextN / MTP / draft), low-bit KV cache, new quant formats — are usable on day 0, with zero compiler knowledge on your part beyond producing the binary (and often not even that).
---
- Node.js 22 or newer — enforced at startup with a clear message. <https://nodejs.org> - Windows, macOS, or Linux. - A GPU is recommended but not required — a CPU build is provisioned as a fallback. - On Windows, the first time the auto-downloaded llama-server runs, SmartScreen/Defender may prompt (it's an upstream binary). Allow it once.
---
npx turbollm
npm install -g turbollm turbollm ```
On first run the daemon:
1. Detects your GPU and downloads a matching llama-server build (CUDA for NVIDIA, ROCm for AMD, Metal for Apple, SYCL for Intel, Vulkan otherwise — with a CPU fallback). 2. Starts on <http://127.0.0.1:6996> and opens your browser. 3. Drops you on the Chat screen, ready to load a model.
Then open Models, download or pick a GGUF, click Load, and start chatting. Stop the daemon any time with Ctrl+C.
---
```bash
Everything lives under ~/.turbollm/ on every OS — config.json, the SQLite chat database, downloaded engines, models cache, and logs. Back it up or delete it to reset. Use --config <file> to point at an alternate config (its directory becomes the data dir).
---
turbollm # start on :6996, open browser
turbollm --port 9000 # listen on a specific port
turbollm --no-open # start without opening a browser
turbollm --addr 0.0.0.0:6996 # bind all interfaces (LAN sharing)
turbollm --stop # stop a running daemon (any terminal)
turbollm launch claude # start Claude Code (auto-loads a model if none is running)
turbollm launch claude --model qwen3-8b # load a specific model, then launch| Flag | Description |
|---|---|
--port <n> | Listen on a specific port (default: 6996) |
--addr <host:port> | Full host:port override, e.g. 0.0.0.0:6996 for LAN sharing |
--no-open | Start without opening a browser window |
--config <file> | Path to a custom config file |
--stop | Stop a running TurboLLM daemon (reads ~/.turbollm/daemon.pid) and exit |
--help, -h | Show usage and exit |
turbollm launch claude also accepts --model <key|name> to load a specific model before launching; without it, an already-loaded model is used, or the last-used / first model is auto-loaded.
---
Same GPU (RTX 5070 Ti 16 GB), same model, same 200K context — measured generation speed. TurboLLM is faster than LM Studio on the very same official llama.cpp, and faster still when you run a community fork LM Studio can't.
① On official llama.cpp, TurboLLM is faster. It auto-provisions a GPU-native engine build (CUDA 13 for Blackwell here) and tunes expert-offload to the layer, so at the same KV-cache quant it beats LM Studio's bundled runtime:
| Qwen3.6-35B-A3B · 200K | TurboLLM | LM Studio | Speed-up |
|---|---|---|---|
official llama.cpp — q4_0 | **74.7 t/s** | 61.0 t/s | **1.2×** |
official llama.cpp — q8_0 | **72.3 t/s** | ~66 t/s\* | **1.1×** |
② Run a faster engine and pull far ahead. Because TurboLLM runs any engine, you can drop in the TurboQuant fork — a llama.cpp fork with a low-bit turbo4 KV cache that LM Studio simply can't load — in one click. On a large-KV model it delivers q8_0-level quality at more than double the speed:
| Qwen3.6-27B · 200K · matched quality | TurboLLM + TurboQuant | LM Studio | Speed-up |
|---|---|---|---|
turbo4 vs q8_0 | **24.6 t/s** | 11.4 t/s | **2.2×** |
Same run, 1.7× faster prefill too (1288 vs 757 tok/s).
<sub>\*LM Studio's q8_0 mildly spilled VRAM at its best offload. A low-bit KV cache helps most when the cache is large; TurboLLM's auto-tuner and on-screen measured t/s pick the fastest engine + config for each model, so you don't have to.</sub>
---
Focused on the differences that matter — all four are good tools, and the others move fast. Marks reflect mid-2026; verify the moving rows against each tool's current docs.
| **TurboLLM** | LM Studio | Ollama | Open WebUI | |
|---|---|---|---|---|
| Run **any engine / community forks** | ✅ | ❌ llama.cpp/MLX only | ❌ hidden | ❌ frontend |
| **Benchmark-based auto-tune** of launch flags | ✅ | ◐ basic offload | ◐ basic offload | ❌ |
| **Measured** t/s in the model list | ✅ | ◐ per-run | ◐ --verbose | ❌ |
**Anthropic** API (/v1/messages) → Claude Code | ✅ | ✅ 0.4.1+ | ✅ v0.14+ | ❌ |
| OpenAI-compatible API | ✅ | ✅ | ✅ | ◐ proxy |
| Auto-load the requested model / multi-model pool | ✅ | ✅ JIT | ✅ | ❌ |
| Use existing model folders (no re-download) | ✅ | ◐ import | ◐ import | ❌ frontend |
| Speculative decoding (draft / MTP) | ✅ | ✅ | ◐ env flag | ❌ |
| Web UI from any LAN device | ✅ | ❌ | ❌ | ✅ |
| **Lightweight** (no Electron / no Python) | ✅ npm | ❌ Electron | ✅ Go | ❌ Python |
| Offline-first · **no telemetry** | ✅ | ◐ analytics on by default | ✅ | ✅ |
LM Studio and Ollama both added Anthropic /v1/messages endpoints in 2026, so the API rows are now parity — Claude Code works against any of them. TurboLLM's durable edges are any engine including community forks, benchmark-based auto-tuning with a VRAM-fit verdict + measured t/s before you commit, and zero telemetry.
Prefer Open WebUI's chat breadth? It works great pointed at TurboLLM's OpenAI endpoint.
---
- TurboLLM requires Node.js 22 or newer — upgrade Node: <https://nodejs.org>. - Model won't load / OOM — pick a smaller quant (the VRAM verdict warns you), lower GPU offload, or close other GPU apps. Failures surface in the Engines screen with the engine log. - Windows Defender / SmartScreen prompt — that's the upstream llama-server binary on first run; allow it once. - Port already in use — turbollm --port 9000. - Slow generation — open the model's load params; ensure GPU offload is high and flash attention / NextN are on for supported models.
---
高性能本地AI模型部署工具
该工具未明确声明开源协议，商业使用前请联系原作者确认授权范围，避免侵权风险。
AI Skill Hub 为第三方内容聚合平台，本页面信息基于公开数据整理，不对工具功能和质量作任何法律背书。
建议在沙箱或测试环境中充分验证后，再部署至生产环境，并做好必要的安全评估。
总体来看，TurboLLM 是一款质量优秀的AI工具，在同类工具中具备一定竞争力。AI Skill Hub 将持续追踪其更新动态，建议收藏备用，结合自身场景选择合适时机引入使用。
| 原始名称 | TurboLLM |
| Topics | AIGPUTypeScript |
| GitHub | https://github.com/mohitsoni48/TurboLLM |
| 语言 | TypeScript |
收录时间：2026-06-27 · 更新时间：2026-06-27 · License：未公布 · AI Skill Hub 不对第三方内容的准确性作法律背书。