个人LLM炼金术

Prerequisites

• macOS on Apple Silicon
• Node.js ≥ 18
• mlx_lm.server (from mlx-lm) — text-only MLX models
• mlx_vlm.server (from mlx-vlm) — vision/multimodal MLX models; optional if you never run VLMs
• llama-server (from llama.cpp)
• hf (from huggingface_hub) — only required for athanor pull

Run athanor doctor at any point to verify all four are on your PATH.

Agent-assisted setup

If you use an AI coding agent (Claude Code, Cursor, Aider, etc.), the fastest path is to open this repo in the agent and ask it to set athanor up for you. AGENTS.md has an Onboarding a user section written for that case: it tells the agent how to install the CLI (npm start vs npm link), run athanor doctor, install any missing runtime binaries, profile the host (Apple Silicon check, unified memory via vm_stat, HF cache size), and pick a starter model sized for the machine.

A minimal prompt, once the repo is open:

Set up athanor on this machine. Profile what I have, install anything missing, and suggest a starter model I can actually run.

The rest of this README walks the same path manually.

Setup

Install the three runtime helpers athanor shells out to. All three land on your PATH and can be verified with athanor doctor.

copy or symlink build/bin/llama-server onto your PATH

Verify:

via uvx — runs the latest release on demand, no install step

uvx hf --help

via pip (the `hf` entry point ships with huggingface_hub ≥ 0.34)

pip install -U huggingface_hub

Verify:

Quick start

Full quick start

```bash npm install

or with a plain venv

python3 -m venv ~/.venvs/mlx && source ~/.venvs/mlx/bin/activate pip install -U mlx-lm

Verify:

MLX requires Apple Silicon and macOS 13.5+. Models are downloaded to ~/.cache/huggingface/hub on first use.

optional: log in if you need access to gated/private repos

hf auth login ```

Configuration

~/.athanor/config.json. Missing fields fall back to these defaults:

{
  "portRange": { "min": 8081, "max": 8099 },
  "enablePiSync": true,
  "modelDirs": {
    "mlx": "~/.cache/huggingface/hub",
    "llama": "~/.models"
  },
  "mlx": {
    "prefillStepSize": 512,
    "promptCacheSize": 32768,
    "decodeConcurrency": 1
  },
  "llama": {
    "nGpuLayers": 999,
    "threads": 8,
    "ctxSize": 32768,
    "batchSize": 512,
    "ubatchSize": 256,
    "parallel": 1
  },
  "supervisor": {
    "policy": "single-active",
    "maxConcurrent": 1,
    "startupTimeoutMs": 120000,
    "healthPollIntervalMs": 500
  },
  "controlApi": {
    "enabled": false,
    "port": 8079,
    "host": "127.0.0.1"
  },
  "router": {
    "enabled": true,
    "port": 8080,
    "host": "127.0.0.1",
    "drainTimeoutMs": 30000
  }
}

inspect effective config, launch command, and running state

athanor show qwen-32b

Environment variables

• ATHANOR_HOME — overrides ~/.athanor. Useful for running multiple profiles side by side or for tests.
• PI_HOME — overrides ~/.pi.

Control API (optional)

When controlApi.enabled is true, athanor exposes a small local HTTP server (default 127.0.0.1:8079) that other tools can drive:

GET  /status                     running instances + registry summary
POST /activate      { "id": "<id|slug>" }   start a model (respects supervisor policy)
POST /deactivate    { "id": "<id|slug>" }   stop a model

This is off by default. Enable it only on trusted machines.

hf (HuggingFace CLI — only for `athanor pull`)

Needed only if you want athanor to download new models from the Hub. Skip if you'll populate ~/.cache/huggingface/hub by other means (e.g. mlx_lm.server --model <repo> auto-downloads on first use).

The Hugging Face Hub team has replaced the legacy huggingface-cli command with a new CLI called hf. Athanor invokes hf download. The recommended install path is the standalone installer, which drops a self-contained hf binary onto your PATH without touching system Python:

curl -LsSf https://hf.co/cli/install.sh | bash

Alternatives:

```bash

CLI reference

athanor                          launch the TUI
athanor scan                     rescan model dirs and update registry
athanor ls                       list registry entries (with live status)
athanor status                   list running instances
athanor show     <id|slug>       inspect a model: runtime, effective config, launch command
athanor snippet  <id|slug>       generate OpenAI API and pi-agent integration code snippets
athanor start    <id|slug>       start a model
athanor stop     [<id|slug>|--all]stop one or all
athanor restart  <id|slug>       stop + start
athanor logs     <id|slug> [-n N] tail last N lines of a running model's log
athanor pull     <repo> [--file F] [--revision R]
                                 download from HuggingFace and register
athanor search   [q] [--mlx|--gguf|--any] [--author A] [--sort S] [--limit N]
                                 search the HuggingFace Hub
athanor trending [--mlx|--gguf] [--limit N]
                                 top trending MLX/GGUF models
athanor preset   <slug> show|set k=v...|unset k...|clear|apply <recipe>
                                 view or modify a model's preset
athanor recipes                  list built-in + user recipes and tunable keys
athanor flavor   <slug> lm|vlm   force MLX runtime flavor (lm = mlx_lm, vlm = mlx_vlm)
athanor expose    <id|slug>      include in pi-agent catalog
athanor hide      <id|slug>      remove from pi-agent catalog
athanor rm       <id|slug>       remove from registry (must be stopped)
athanor sync                     manually rewrite pi catalog
athanor config                   print resolved config and its path
athanor doctor                   check that required binaries are on PATH and show installed versions
athanor doctor --check-updates   also compare installed versions with latest available and print upgrade hints

<id|slug> accepts either the canonical id or the short slug.

Troubleshooting

• athanor start hangs or times out. Check ~/.athanor/logs/<slug>-<pid>.log. Most startup failures are the runtime itself complaining (missing weights, wrong quant, out of memory). Raise supervisor.startupTimeoutMs for very large models.
• port already in use. Another process is on the model's stable port. Either stop it, or edit the entry's port in ~/.athanor/models.json and restart.
• Pi-agent can't see a new model. Make sure it's exposed (CLI: athanor expose <slug>) and run athanor sync. Confirm ~/.pi/agent/models.json contains the expected athanor provider shape (per-model when router.enabled is false, athanor-mlx / athanor-llama aggregators when true — the default), then open /model in pi (the file reloads on open). If you upgraded athanor and llama model ids changed, restart the model after syncing.
• Models from other tools disappeared from pi. They shouldn't — athanor only rewrites providers whose name starts with athanor-. If this happens, open an issue with the before/after of ~/.pi/agent/models.json.
• Stale PID / router state. If a child or detached router crashed without athanor noticing, reopening athanor or running athanor sync / athanor status will reconcile persisted state and clear dead router metadata opportunistically. If a model port is still held, run athanor stop <slug> (a no-op when nothing is live) then athanor start <slug>.
• doctor reports a missing binary. Install mlx_lm, mlx_vlm, llama.cpp, or huggingface_hub, or adjust your shell's PATH. mlx_vlm.server is only needed if you plan to run VLM models; athanor start on a VLM entry will fail with a clear error if it's missing.
• doctor --check-updates reports update available. Follow the printed one-line hint. Today the built-in hints cover uv-managed Python tools (uv tool upgrade mlx-lm, uv tool upgrade mlx-vlm, uv tool upgrade hf) and Homebrew's llama.cpp formula (brew upgrade llama.cpp).