One local server.
Three APIs your tools already speak.

Getting started

Start the server from the MLX Core app (it runs whenever a model is loaded) or from the CLI:

mlx-serve serve          # serves everything under ~/.mlx-serve/models, loads on demand
mlx-serve run gemma4     # or: download one model, serve it, and chat

The default port is 11234 (--port to change it). Models are addressed by name — the model field accepts a model's directory name, an org/repo id, or an Ollama-style short name with tag. A request naming a discovered-but-unloaded model loads it on the fly.

curl http://localhost:11234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-e4b-it-8bit",
    "messages": [{"role": "user", "content": "Why is the sky blue?"}],
    "stream": true
  }'

Or with the OpenAI SDK — just point base_url at your Mac:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11234/v1", api_key="unused")
r = client.chat.completions.create(
    model="gemma-4-e4b-it-8bit",
    messages=[{"role": "user", "content": "Hello!"}])
print(r.choices[0].message.content)

OpenAI-compatible endpoints

Every response carries usage.prompt_tokens_details.cached_tokens, so you can see what the prefix cache saved. stream_options: {"include_usage": true} adds the usage chunk at the end of a stream. When a reply is cut because the model went in circles, finish_details: {"type": "repetition_loop"} sits beside finish_reason instead of the cut being reported as a plain length stop.

Request extensions beyond OpenAI

All optional; every endpoint stays spec-compatible without them.

Sampling defaults resolve in this order: request body → server launch flags → the model's own generation_config.json → engine defaults. Clients that omit temperature/top_p/top_k get what the model's authors recommend.

Anthropic Messages API

POST /v1/messages implements the Anthropic Messages API — typed content blocks, tool_use / tool_result, thinking blocks, and the full SSE event lifecycle (message_start, content_block_delta, …). It's what lets Claude Code run against your local model:

export ANTHROPIC_BASE_URL=http://localhost:11234
export ANTHROPIC_API_KEY=unused
claude   # Claude Code now talks to your Mac

OpenAI Responses API

Ollama-compatible API

Tools built for Ollama — Raycast, Obsidian, Enchanted, Open WebUI, ollama-python/js — connect unchanged (the swap, in depth). Run on Ollama's port and nothing needs configuring at all: mlx-serve serve --port 11434.

Media generation

Each media endpoint accepts "stream": true for SSE progress events ending in a base64 complete payload; without it the response is the finished artifact. Load and unload media models like any other (below) — chat and media coexist in one process.

POST/v1/images/generations

FLUX.2-klein (4B and 9B), Krea-2-Turbo and Mage-Flow Turbo / Edit (deep dive). Returns base64 PNG.

POST/v1/images/edits

The OpenAI SDK's edit shape, so client.images.edit(image=…, prompt=…) works unchanged against any edit-capable model you have loaded. This one is multipart/form-data, not JSON, and it is pure translation into the mode:"edit" body above — there is no second inference path.

client.images.edit(
    model="mage-flow-edit-turbo-8bit",
    image=[open("room.png","rb"), open("chair.png","rb")],   # repeated image[] = multi-reference
    prompt="put the chair from image 2 into image 1")

Accepted fields: model, prompt, image / image[] (up to 4), size. Anything we can't honor is a named 400 rather than a silent no-op: mask (the editors are maskless), n > 1, response_format: "url", an output_format other than png, and stream.

POST/v1/audio/speech

Qwen3-TTS (voice cloning) or Kokoro-82M (54 built-in voices), whichever is loaded (deep dive). Returns WAV bytes. The two backends share the endpoint but not the controls, and asking one for the other's control is a named 400 rather than a quietly plain-voiced answer.

POST/v1/audio/music-generations

Text-to-music, WAV out (deep dive): ACE-Step (48 kHz stereo, fast) or MiniMax Music 3 (sings full songs at 44.1 kHz, up to six minutes). On Music 3, lyrics is required and the musical-steering fields below don't exist; put tempo and key in the caption.

POST/v1/video/generations

One endpoint, two backends: LTX-Video 2.3 / 2.5 (text, image or audio to video with synced sound; on 2.5, "decoder": "diffusion" picks its sharper diffusion decoder) and MiniMax-H3 / Hailuo 3.0 (video and its stereo soundtrack denoised jointly in one pass, long clips chained via chain_windows). Whichever is loaded serves the request, and the field a backend can't honor is a named 400 rather than a silently dropped setting (deep dive).

The response is raw frames plus raw audio, not a container: {"frames": N, "width", "height", "fps": 24, "format": "rgb8", "data": "<base64>"}, and when the model produced sound, audio_format: "pcm_s16le", audio_channels: 2, audio_sample_rate and audio_data beside it. Mux to MP4 on your side. These runs take minutes, so use "stream": true for progress and give your client a generous timeout.

Opt-in per-step preview (issue #208): "preview": true with "stream": true attaches a JPEG to each denoise progress event — Latent2RGB of the predicted clean latent, not a full VAE decode. Default off, so existing clients keep the {type, stage, step, total} shape. preview_frames (default 1, max 8) packs extra temporal slices as a horizontal filmstrip; preview_max_side (default 256, 0 = latent native) caps the long edge. Cached-velocity H3 steps emit progress without an image. complete is still the full rgb8 + PCM.

LTX-Video

MiniMax-H3 — narrower on purpose: no CFG scale and no pipeline mode (it is CFG-distilled and single-pipeline), and frame counts live on its own ladder. New in v26.8.2. It ships as two partitions that carry the same files and differ only in which conditioning they were trained for, so the server reads the pack's declared tasks and refuses what that build would ignore: FL2VA does keyframes and chained windows, REF2VA does references.

POST/v1/3d/generations

Hunyuan3D-2.1 — one photo in, a GLB mesh out; optional full-PBR texturing (deep dive). New in v26.7.2.

Response: {"created": …, "format": "glb", "data": "<base64 GLB>"} — standard glTF, opens anywhere.

Model management

Requests are queued per model and batch-decode together where the architecture allows; a media generation runs on the same GPU and briefly pauses chat decode — one machine, one memory budget, no surprises.

--model-dir is repeatable, so a server can serve several folders at once. They merge first-wins, and every one of them shows up in /v1/models.

Utility endpoints

Metrics & auth

Both are opt-in and independent. Start the server with --metrics, --api-key, or neither — the app exposes the same two switches in Settings.

Recording happens once per request, never per token, so the metrics have no measurable effect on tokens/sec.

With --api-key <token> set, every request arriving from another machine must present the key — the OpenAI, Anthropic, and Ollama APIs, the index page, and the endpoints above. Any of these work:

curl http://mac.local:11234/v1/models -H "Authorization: Bearer <key>"
curl http://mac.local:11234/v1/models -H "x-api-key: <key>"
curl "http://mac.local:11234/v1/models?api_key=<key>"

Requests from the machine itself are trusted and need no key, so the app, your editor, and local scripts are unaffected — the key guards only what is reachable off the box. /health stays open either way. Because localhost is exempt, curl localhost will never show you a 401; test from another machine.