Otari LLM网关

1) Install

uv venv
source .venv/bin/activate
uv sync --dev

Docker

The gateway image is published on Docker Hub.

Run with docker compose (gateway + PostgreSQL)

cp config.example.yml config.yml
docker compose up -d

Run with docker only

docker run --rm \
  -p 8000:8000 \
  -v "$(pwd)/config.yml:/app/config.yml:ro" \
  mzdotai/otari:latest \
  gateway serve --config /app/config.yml

Gateway will be available at http://localhost:8000.

Quickstart

2) Configure

cp config.example.yml config.yml

Edit config.yml and set at least:

• master_key
• one provider credential in providers (for example openai.api_key)

First request (OpenAI SDK)

On startup, the gateway can bootstrap an API key in logs. Export it as GATEWAY_API_KEY, then call the gateway as an OpenAI-compatible server:

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GATEWAY_API_KEY"],
    base_url="http://localhost:8000/v1",
)

response = client.chat.completions.create(
    model="openai:gpt-4o",
    messages=[{"role": "user", "content": "Hello from gateway"}],
)

print(response.choices[0].message.content)

API surface

• GET /health
• POST /v1/chat/completions
• POST /v1/embeddings
• POST /v1/moderations
• GET /v1/models
• POST/GET /v1/keys
• POST/GET /v1/users
• POST/GET /v1/budgets
• GET /v1/messages
• GET /v1/pricing

Full schema: docs/public/openapi.json

Useful CLI commands

uv run gateway init-db --config config.yml
uv run gateway migrate --config config.yml
uv run gateway migrate --config config.yml --revision <rev>
uv run python scripts/generate_openapi.py --check