苹果芯片AI工作流

2. Install Tooling

Minimum tools:

• Node.js 16.7 or newer
• npm
• CMake 3.27 or newer
• Ninja
• a C++20 compiler

Recommended on macOS:

xcode-select --install
brew install cmake ninja node
npm ci
npx playwright install

Recommended on Windows:

npm ci
npx playwright install

On Windows, run native CMake commands from a Visual Studio Developer shell when available.

3. Configure And Build

macOS/Linux:

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

Windows PowerShell:

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

If Ninja is not available, CMake may use your platform default generator. Keep the same build directory.

chat (replaces `ollama run <model>` style usage)

curl -s http://127.0.0.1:8080/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model":"qwen2.5-coder-7b", "messages":[{"role":"user","content":"fizzbuzz in python"}] }'

6. Serve OpenAI-Compatible Endpoint (Local LLM, Ollama-Compatible)

US4 V6 ships an OpenAI-shape HTTP endpoint that drop-in replaces Ollama for any client expecting /v1/chat/completions, /v1/completions, /v1/models, or /v1/embeddings. Chat is served by mlx_lm.server (managed child process); embeddings are served in-process by mlx-embeddings. Single-file Python sidecar at scripts/openai_serve.py. No FastAPI, no uvicorn.

Two ways to run it: the Python sidecar directly (no C++ build required — fastest path, recommended for local LLM use) or the C++ CLI wrapper (us4-cli serve, identical contract once the native build exists).

6.1 Python sidecar (recommended — no C++ build)

One-time setup. Use a project venv to avoid externally-managed-environment on Homebrew Python:

python3 -m venv .venv
.venv/bin/pip install -r scripts/requirements-serve.txt

Run with defaults (chat + embeddings, bind 127.0.0.1:8080, child mlx-lm on 8081). Always invoke the venv interpreter explicitly — python3 from the system PATH will not see MLX:

.venv/bin/python scripts/openai_serve.py

When running the Python sidecar directly, configure it with environment variables:

Example: pick a smaller 3B chat model on a memory-constrained M1 8 GB:

US4_SERVE_CHAT_MODEL=mlx-community/Qwen2.5-Coder-3B-Instruct-4bit \
US4_SERVE_PORT=8080 \
.venv/bin/python scripts/openai_serve.py

First start downloads model weights from HuggingFace into ~/.cache/huggingface/ (7B 4-bit MLX ≈ 4 GB, 3B 4-bit ≈ 1.7 GB). Subsequent starts reuse the cache.

Example: front an already-running Ollama daemon (chat goes through Ollama, embeddings stay MLX-local). Useful when you want to reuse models already pulled into Ollama without re-downloading the MLX variant from HuggingFace:

```bash

any OpenAI SDK / langchain / litellm / continue.dev now hits us4-v6 locally

`simplicio-cli` consumes the same env vars:

6.5 Hardware sizing on Apple Silicon

The us4-v6 MLX path is unified-memory aware: 4-bit MLX weights occupy real RAM only once and are shared between CPU and GPU without copies. Compared to GGUF/Ollama on the same Apple Silicon machine, RAM headroom is meaningful on small machines:

Override US4_SERVE_CHAT_MODEL to match the box you are on. Pulling a 7B model on an 8 GB machine is feasible but expect macOS to swap under load — prefer a 3B for sustained chat.

6.6 Troubleshooting

Full contract (endpoints, request/response shapes, env knobs, exit codes, security posture) lives in .specs/runtime/SERVE-OPENAI.md.