# Server Module

## Purpose and Architecture Position

The Server Module provides a production-ready FastAPI REST server that exposes AbstractCore's capabilities through OpenAI-compatible HTTP endpoints. It acts as a universal LLM gateway with comprehensive media processing, multi-agent tool call syntax support, and provider-agnostic model access.

**Architecture Position**: The Server Module sits at the top of the AbstractCore stack, providing HTTP/REST access to all underlying capabilities. It transforms AbstractCore's Python API into web-accessible endpoints for integration with any application, regardless of programming language.

## Quick Reference

### API Endpoints Quick Reference

| Endpoint | Method | Purpose | Key Parameters |
|----------|--------|---------|----------------|
| `/health` | GET | Server health check | - |
| `/v1/models` | GET | List/filter available models | `provider`, `input_type`, `output_type`, `base_url`, `api_key` |
| `/providers` | GET | List providers with metadata | `include_models` |
| `/v1/vision/providers/` | GET | AbstractVision provider model catalog | `task`, `provider`, `include_models`, `base_url`, `api_key` |
| `/v1/audio/voices` | GET | AbstractVoice voice/profile catalog | `provider`, `model`, `providers_only`, `base_url`, `api_key` |
| `/v1/audio/speech/models` | GET | AbstractVoice TTS model catalog | `provider`, `base_url`, `api_key` |
| `/v1/audio/speech/providers` | GET | AbstractVoice TTS provider catalog | `base_url` |
| `/v1/audio/transcriptions/models` | GET | AbstractVoice STT model catalog | `provider`, `base_url`, `api_key` |
| `/v1/audio/transcriptions/providers` | GET | AbstractVoice STT provider catalog | `base_url` |
| `/v1/voice/clone/providers` | GET | AbstractVoice voice clone provider catalog | `base_url` |
| `/v1/chat/completions` | POST | Chat completions | `model`, `messages`, `stream` |
| `/v1/embeddings` | POST | Generate embeddings | `model`, `input` |
| `/v1/responses` | POST | OpenAI Responses API | `model`, `input` |
| `/v1/images/generations` | POST | Image generation (optional) | `prompt`, `model`, `provider`, `base_url`, `width`, `height`, `size` |
| `/{provider}/v1/images/generations` | POST | Provider-scoped image generation | `model` (no prefix), `prompt`, `base_url` |
| `/v1/images/edits` | POST | Image editing (optional) | `prompt`, `image`, `model`, `provider`, `base_url`, `mask`, `size` |
| `/{provider}/v1/images/edits` | POST | Provider-scoped image editing | `model` (no prefix), `prompt`, `image`, `base_url` |
| `/v1/videos/generations` | POST | Text-to-video generation (optional) | `prompt`, `model`, `provider`, `base_url`, `width`, `height`, `fps`, `num_frames`, `steps`, `guidance_scale`, `guidance_2` |
| `/{provider}/v1/videos/generations` | POST | Provider-scoped text-to-video | `model` (no prefix), `prompt`, `base_url` |
| `/v1/videos/edits` | POST | Image-to-video generation (optional) | `prompt`, `image`, `model`, `provider`, `base_url`, `fps`, `num_frames`, `steps`, `guidance_scale`, `guidance_2` |
| `/{provider}/v1/videos/edits` | POST | Provider-scoped image-to-video | `model` (no prefix), `prompt`, `image`, `base_url` |
| `/v1/vision/models` | GET | Available vision model catalog | `task`, `provider`, `base_url`, `api_key` |
| `/v1/vision/jobs/*` | GET/POST | Async image/video jobs | `job_id`, `consume`, media parameters, `progress.last_event` |
| `/v1/audio/transcriptions` | POST | Speech-to-text (optional) | `file`, `provider`, `model`, `base_url` |
| `/{provider}/v1/audio/transcriptions` | POST | Provider-scoped speech-to-text | `model` (no prefix), `file`, `base_url` |
| `/v1/audio/translations` | POST | Audio translations (not yet supported) | - |
| `/v1/audio/speech` | POST | Text-to-speech (optional) | `input`, `provider`, `model`, `voice`, `format`, `profile`, `quality_preset`, `quality`, `base_url` |
| `/{provider}/v1/audio/speech` | POST | Provider-scoped text-to-speech | `model` (no prefix), `input`, `base_url` |
| `/v1/voice/clone` | POST | Voice clone/custom voice extension (optional) | `file`, `provider`, `model`, `tts_model`, `cloning_engine`, `base_url`, `name` |
| `/{provider}/v1/voice/clone` | POST | Provider-scoped voice clone/custom voice extension | `model` (no prefix), `file`, `base_url`, `name` |
| `/v1/audio/music` | POST | Text-to-music (optional) | `prompt`, optional `backend`, `model`, `lyrics`, `duration_s`, `format` (`wav`, `mp3`, `flac`) |
| `/{provider}/v1/chat/completions` | POST | Provider-specific endpoint | `model` (no prefix) |
| `/acore/models/load` | POST | Load a task-specific provider/model runtime | optional `task` (`text_generation` default, `image_generation`, `video_generation`, `text_to_video`, `image_to_video`, `tts`, `stt`), `provider`, `model`, `options`, `pin`, `base_url`, `timeout_s` |
| `/acore/models/loaded` | GET | List currently loaded task-aware runtimes | optional `task`, `provider`, `model` |
| `/acore/models/unload` | POST | Unload a task-specific runtime | `runtime_id` or `provider` + `model`, optional `task`, `base_url`, `options` |
| `/acore/prompt_cache/*` | GET/POST | Prompt-cache control plane on a loaded gateway runtime or proxied AbstractEndpoint | `provider` + `model` or `base_url`, cache operation fields |
| `/acore/blocs/*` | GET/POST | Provider-backed memory-bloc KV artifact control plane for MLX, HuggingFace transformers, and supported HuggingFace GGUF exact-renderer paths | local store by default, or `runtime_id` / `provider` + `model` / `base_url` as required |

Runtime note: omitted `task` on `/acore/models/load` keeps the existing text-generation runtime behavior. Non-text tasks route to capability-owned load/list/unload where supported: `image_generation`, `video_generation`, `text_to_video`, and `image_to_video` use the same AbstractVision backend cache, while `tts` and `stt` use the shared AbstractVoice capability core. `loaded_new` is optional event metadata for the *load call* (did this call likely transition `loaded` from false to true?); it is not the loaded state itself.

### Common Request Patterns

| Pattern | Endpoint | Key Fields | Example |
|---------|----------|------------|---------|
| **Simple Chat** | `/v1/chat/completions` | `model`, `messages` | Text conversation |
| **Streaming** | `/v1/chat/completions` | `stream: true` | Real-time responses |
| **Vision** | `/v1/chat/completions` | `content: [text, image_url]` | Image analysis |
| **Media Catalogs** | `/v1/vision/providers/`, `/v1/audio/voices` | `task`, `provider`, `base_url`, `api_key` | Populate image model and voice dropdowns |
| **Image Generation** | `/v1/images/generations`, `/{provider}/v1/images/generations` | `prompt`, optional `model`/`provider`/`base_url` | Create images (optional) |
| **Image Editing** | `/v1/images/edits`, `/{provider}/v1/images/edits` | `prompt`, `image`, optional `model`/`provider`/`base_url` | Edit images (optional) |
| **Video Generation** | `/v1/videos/generations`, `/v1/videos/edits` | `prompt`, optional source `image`, optional `model`/`provider`/`base_url` | Create videos (optional) |
| **Speech-to-Text** | `/v1/audio/transcriptions`, `/{provider}/v1/audio/transcriptions` | `file`, optional `model`/`provider`/`base_url` | Transcribe audio (optional) |
| **Audio Translations** | `/v1/audio/translations` | `file` | Translate audio (not yet supported) |
| **Text-to-Speech** | `/v1/audio/speech`, `/{provider}/v1/audio/speech` | `input`, optional `model`/`provider`/`base_url` | Generate audio (optional) |
| **Voice Clone** | `/v1/voice/clone`, `/{provider}/v1/voice/clone` | `file`, optional `model`/`provider`/`base_url` | Create a compatible custom voice (optional) |
| **Text-to-Music** | `/v1/audio/music` | `prompt` | Generate music/audio through optional AbstractMusic backends; `abstractmusic>=0.1.13` supports remote ACE Music with `ACEMUSIC_API_KEY` |
| **Documents** | `/v1/chat/completions` | `content: [text, file_url]` | PDF/CSV processing |
| **Tools** | `/v1/chat/completions` | `tools`, `tool_choice` | Function calling |
| **Embeddings** | `/v1/embeddings` | `model`, `input` | Text embeddings |

### Media Support

| Media Type | MIME Types | Max Size | Processing |
|------------|-----------|----------|------------|
| **Images** | `image/*` | 10MB | Vision models |
| **Audio** | `audio/*` | 25MB for `/v1/audio/transcriptions` | Remote STT or capability plugins |
| **Documents** | `application/pdf`, `docx`, `xlsx` | 10MB | Text extraction |
| **Data** | `text/csv`, `application/json` | 10MB | Rendering |
| **Text** | `text/plain`, `text/html` | 10MB | Direct |

## Common Tasks

- **How do I start the server?** → See [Local Development](#local-development)
- **How do I list models?** → See [List Models](#2-list-models)
- **How do I make a chat request?** → See [Chat Completions](#4-chat-completions)
- **How do I process images?** → See [Vision API](#5-vision-api-image-analysis)
- **How do I process documents?** → See [Document Processing](#6-document-processing)
- **How do I generate images?** → See [Image Generation](#image-generation-optional)
- **How do I enable streaming?** → See [Streaming Response](#streaming-response)
- **How do I use function calling?** → See [With Function Calling](#with-function-calling)
- **How do I deploy to production?** → See [Production Deployment](#production-deployment)
- **How do I troubleshoot errors?** → See [Troubleshooting](#troubleshooting)

```
┌─────────────────────────────────────────────┐
│         Server Module (server/)             │  ← You are here
│   (FastAPI REST API, OpenAI-compatible)     │
└────────────────┬────────────────────────────┘
                 │
┌────────────────▼────────────────────────────┐
│       Processing Module (processing/)       │
│  (Specialized NLP operations & workflows)   │
└────────────────┬────────────────────────────┘
                 │
┌────────────────▼────────────────────────────┐
│        Core Layer (core/, providers/)       │
│  (LLM providers, factory, base interfaces)  │
└─────────────────────────────────────────────┘
```

## Component Structure

The Server Module consists of a single, comprehensive FastAPI application:

```
server/
├── __init__.py                    # Module exports
└── app.py                         # FastAPI server (2000+ lines)
```

### Design Philosophy

- **OpenAI Compatibility**: 100% compatible with OpenAI Chat Completions API
- **Universal Gateway**: Access any AbstractCore provider through unified API
- **Media Processing**: Built-in support for images, documents, PDFs, CSV, Excel, etc.
- **Tool Call Flexibility**: Auto-detection and conversion of tool call syntaxes
- **Clean Delegation**: Minimal server logic, delegates to AbstractCore
- **Production-Ready**: Comprehensive error handling, logging, CORS support
- **Zero Configuration**: Works out-of-the-box with sensible defaults

---

## API Endpoints

### 1. Health Check

**Endpoint**: `GET /health`

**Purpose**: Server health and version information.

**Response**:
```json
{
  "status": "healthy",
  "version": "2.5.2",
  "timestamp": "2025-11-06T12:00:00Z"
}
```

**Usage**:
```bash
curl http://localhost:8000/health
```

---

### 2. List Models

**Endpoint**: `GET /v1/models`

**Purpose**: List all available models across all providers.

**Query Parameters**:
- `input_type` (optional): Filter by input capability (`text`, `image`, `audio`, `video`)
- `output_type` (optional): Filter by output capability (`text`, `embeddings`)
- `provider` (optional): Filter by specific provider

**Response**:
```json
{
  "object": "list",
  "data": [
    {
      "id": "openai/gpt-4o-mini",
      "object": "model",
      "created": 1234567890,
      "owned_by": "openai"
    },
    {
      "id": "ollama/llama3.2:3b",
      "object": "model",
      "created": 1234567890,
      "owned_by": "ollama"
    }
  ]
}
```

**Usage**:
```bash
# List all models
curl http://localhost:8000/v1/models

# List models that can analyze images (vision models)
curl http://localhost:8000/v1/models?input_type=image

# List embedding models
curl http://localhost:8000/v1/models?output_type=embeddings

# List text-only models that generate text
curl http://localhost:8000/v1/models?input_type=text&output_type=text

# List Ollama vision models
curl http://localhost:8000/v1/models?provider=ollama&input_type=image

# Combine multiple filters
curl http://localhost:8000/v1/models?provider=openai&input_type=image&output_type=text
```

**Python Client**:
```python
import requests

# List all models
response = requests.get("http://localhost:8000/v1/models")
models = response.json()["data"]

# Filter by capabilities
response = requests.get("http://localhost:8000/v1/models?input_type=image")
vision_models = response.json()["data"]

response = requests.get("http://localhost:8000/v1/models?output_type=embeddings")
embedding_models = response.json()["data"]
```

---

### Capability Catalogs

Use these routes when a client needs image model, TTS model, or voice/profile
dropdowns. They use the AbstractVision/AbstractVoice plugin boundary and keep
`/v1/models` focused on LLM and embedding provider models.

```bash
curl http://localhost:8000/v1/vision/providers/?task=text_to_image
curl http://localhost:8000/v1/vision/models
curl http://localhost:8000/v1/audio/voices
curl http://localhost:8000/v1/audio/speech/models
curl http://localhost:8000/v1/audio/speech/providers
curl http://localhost:8000/v1/audio/transcriptions/models
curl http://localhost:8000/v1/audio/transcriptions/providers
curl http://localhost:8000/v1/voice/clone/providers
```

`base_url` can target a local or remote OpenAI-compatible media endpoint. It
uses the same loopback-by-default / allowlist policy as generation routes.
Provider-key overrides use `X-AbstractCore-Provider-API-Key`.

---

### 3. List Providers

**Endpoint**: `GET /providers`

**Purpose**: List all registered providers with their metadata and available models.

**Response**:
```json
{
  "providers": [
    {
      "name": "openai",
      "description": "OpenAI Language Models",
      "requires_api_key": true,
      "api_key_env": "OPENAI_API_KEY",
      "supported_features": [
        "streaming",
        "function_calling",
        "vision",
        "structured_output"
      ],
      "models": ["gpt-4o-mini", "gpt-4o", "gpt-4-turbo", ...],
      "installation": "pip install openai"
    },
    {
      "name": "ollama",
      "description": "Local LLM Serving with Ollama",
      "requires_api_key": false,
      "supported_features": [
        "streaming",
        "function_calling",
        "structured_output"
      ],
      "models": ["llama3.2:3b", "qwen3:4b", "gemma3:1b-it-qat", ...],
      "installation": "Install from https://ollama.com/"
    }
  ],
  "total_providers": 6,
  "total_models": 137,
  "registry_version": "2.0"
}
```

**Usage**:
```bash
curl http://localhost:8000/providers
```

---

### 4. Chat Completions

**Endpoint**: `POST /v1/chat/completions`

**Purpose**: Generate chat completions using any AbstractCore provider.

**Request Schema**:
```json
{
  "model": "openai/gpt-4o-mini",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "What is the capital of France?"
    }
  ],
  "temperature": 0.7,
  "max_tokens": 2048,
  "top_p": 1.0,
  "stream": false,
  "tools": [...],
  "tool_choice": "auto",
  "stop": ["END"],
  "seed": 12345,
  "frequency_penalty": 0.0,
  "presence_penalty": 0.0,
  "agent_format": "auto",
  "base_url": null
}
```

**Key Parameters**:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model` | string | **required** | Model ID in `provider/model` format |
| `messages` | array | **required** | Conversation history |
| `temperature` | float | 0.7 | Sampling temperature (0-2) |
| `max_tokens` | integer | null | Max tokens to generate |
| `top_p` | float | 1.0 | Nucleus sampling (0-1) |
| `stream` | boolean | false | Enable streaming responses |
| `tools` | array | null | Available function tools |
| `tool_choice` | string/object | "auto" | Tool calling strategy |
| `agent_format` | string | null | Tool syntax format |
| `api_key` | string | null | Deprecated/disabled in request bodies. Use server-side provider config or `X-AbstractCore-Provider-API-Key` for a provider override. |
| `base_url` | string | null | Custom API endpoint URL |

**Response (Non-Streaming)**:
```json
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "openai/gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 10,
    "total_tokens": 30
  }
}
```

**Response (Streaming)**:
```
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"openai/gpt-4o-mini","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"openai/gpt-4o-mini","choices":[{"index":0,"delta":{"content":" capital"},"finish_reason":null}]}

data: [DONE]
```

**Usage Examples**:

**Server authentication**:
If `ABSTRACTCORE_AUTH_TOKEN` is configured, every non-health endpoint requires
`Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN`. Authenticated clients can use all
provider keys/endpoints configured on the server. If it is not configured,
either set `ABSTRACTCORE_SERVER_ALLOW_UNAUTHENTICATED=1` for intentional local/dev use, or
pass an explicit upstream provider key via `X-AbstractCore-Provider-API-Key`.
For Swagger UI convenience, when server auth is not configured the server also treats
`Authorization: Bearer <provider-key>` as an upstream provider key. `GET /health`
is always unauthenticated.

Swagger UI at `http://localhost:8000/docs` keeps the normal `Authorize` button
when server auth is enabled. Enter the `ABSTRACTCORE_AUTH_TOKEN` value there; AbstractCore validates the bearer token
before Swagger marks it authorized for `Try it out`.
The dependency-free endpoint index is available at `http://localhost:8000/docs-lite`.
The docs and OpenAPI schema are public by default so the UI can load before authentication;
set `ABSTRACTCORE_SERVER_PROTECT_DOCS=1` to put `/docs`, `/docs-lite`, `/redoc`, and
`/openapi.json` behind server auth too. When server auth is disabled, the
server bearer scheme is omitted from Swagger UI.

Every request body has a Swagger example. Optional aliases that would conflict
when sent together are shown as `null`; the server drops nulls before routing.
Use `base_url` only when you intentionally want to target a local/gateway
OpenAI-compatible endpoint instead of the provider's default API host.

For `/v1/audio/speech`, AbstractCore's Swagger page converts authenticated
binary audio `POST` responses into browser `blob:` URLs before Swagger renders
the player. The example requests `wav` because browser players handle WAV
duration metadata more reliably than MP3 in raw binary previews. If the embedded
player still shows `0:00`, use the response download or save the same request
with curl; the endpoint still returned normal audio bytes.

**Basic Text Chat**:
```bash
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "temperature": 0.7
  }'
```

#### Streaming response
```bash
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "ollama/llama3.2:3b",
    "messages": [{"role": "user", "content": "Tell me a story"}],
    "stream": true
  }'
```

#### With function calling
```bash
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string"},
              "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'
```

**Python Client**:
```python
import requests

response = requests.post(
    "http://localhost:8000/v1/chat/completions",
    json={
        "model": "openai/gpt-4o-mini",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What is the capital of France?"}
        ],
        "temperature": 0.7,
        "max_tokens": 150
    }
)

result = response.json()
print(result["choices"][0]["message"]["content"])
```

**OpenAI Python Client**:
```python
import os
from openai import OpenAI

# Point to AbstractCore server
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key=os.environ["ABSTRACTCORE_AUTH_TOKEN"]
)

response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)
```

**Provider authentication**:
```bash
# Preferred: configure provider keys on the server (e.g. OPENAI_API_KEY).
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

```bash
# Override only the requested upstream provider while still using the server auth token.
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \
  -H "X-AbstractCore-Provider-API-Key: $ANTHROPIC_API_KEY" \
  -d '{
    "model": "anthropic/claude-haiku-4-5",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

```python
# When server auth is not configured, either set ABSTRACTCORE_SERVER_ALLOW_UNAUTHENTICATED=1
# for intentional local/dev use, or provide an upstream provider key explicitly.
response = requests.post(
    "http://localhost:8000/v1/chat/completions",
    headers={"X-AbstractCore-Provider-API-Key": "sk-provider-key"},
    json={
        "model": "openai-compatible/my-model",
        "messages": [{"role": "user", "content": "Hello!"}],
        "base_url": "https://my-custom-endpoint.com/v1",
    },
)
```

Note: provider keys in request bodies remain disabled because those values are routinely logged by clients, proxies, and servers. Select discovery endpoints accept an `api_key` query parameter for tooling/Swagger UI convenience, but headers remain preferred.

---

### 5. Vision API (Image Analysis)

**Endpoint**: `POST /v1/chat/completions` (multimodal messages)

**Purpose**: Analyze images using vision-capable models.

**Request with Image**:
```json
{
  "model": "ollama/qwen2.5vl:7b",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What's in this image?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/jpeg;base64,/9j/4AAQSkZJRg..."
          }
        }
      ]
    }
  ]
}
```

**Supported Image Formats**:
- PNG, JPEG, GIF, WEBP, BMP, TIFF
- Base64 data URLs: `data:image/jpeg;base64,...`
- HTTP(S) URLs: `https://example.com/image.jpg`
- Size limit: 10MB per image

**Usage**:
```bash
# Using base64 encoded image
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ollama/qwen2.5vl:7b",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image"},
        {
          "type": "image_url",
          "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ..."}
        }
      ]
    }]
  }'

# Using HTTP URL
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What do you see?"},
        {
          "type": "image_url",
          "image_url": {"url": "https://example.com/photo.jpg"}
        }
      ]
    }]
  }'
```

**Python with Image**:
```python
import requests
import base64

# Read and encode image
with open("image.jpg", "rb") as f:
    image_base64 = base64.b64encode(f.read()).decode()

response = requests.post(
    "http://localhost:8000/v1/chat/completions",
    json={
        "model": "ollama/qwen2.5vl:7b",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}
                }
            ]
        }]
    }
)

print(response.json()["choices"][0]["message"]["content"])
```

---

### Image/Video Generation (optional)

AbstractCore Server can optionally expose OpenAI-compatible image and video endpoints:
- `POST /v1/images/generations`
- `POST /{provider}/v1/images/generations`
- `POST /v1/images/edits`
- `POST /{provider}/v1/images/edits`
- `POST /v1/images/upscale`
- `POST /{provider}/v1/images/upscale`
- `POST /v1/videos/generations`
- `POST /{provider}/v1/videos/generations`
- `POST /v1/videos/edits`
- `POST /{provider}/v1/videos/edits`
- `POST /v1/vision/jobs/images/generations`
- `POST /v1/vision/jobs/images/edits`
- `POST /v1/vision/jobs/images/upscale`
- `POST /v1/vision/jobs/videos/generations`
- `POST /v1/vision/jobs/videos/edits`

Example async SeedVR2 image upscale job:

```bash
JOB_ID=$(curl -sS -X POST "$BASE/v1/vision/jobs/images/upscale" \
  -H "Authorization: Bearer $TOKEN" \
  -F "provider=mlx-gen" \
  -F "model=AbstractFramework/seedvr2-3b-8bit" \
  -F "image=@./input.png;type=image/png" \
  -F "scale=2x" \
  | python -c 'import json,sys; print(json.load(sys.stdin)["job_id"])')

curl -sS "$BASE/v1/vision/jobs/$JOB_ID" \
  -H "Authorization: Bearer $TOKEN"
```

If `model` is omitted, the server uses its configured AbstractVision/OpenAI-compatible
media default. AbstractCore does not hardcode a local image or video model default.
Provider-scoped routes accept an unprefixed body `model`, matching the chat
route pattern used by `/{provider}/v1/chat/completions`.
Global routes also accept optional `provider` and `base_url` overrides so image
generation, image edits/upscaling, text-to-video, and image-to-video follow the same routing contract as the audio
endpoints and the Python `generate(..., output="image")` path.

**Backends (env vars)**:

- **Default**: `auto` (picks a backend per request)
  - If `model` is omitted, uses the configured AbstractVision/OpenAI-compatible image default
  - If `model` is `diffusers/default`, uses the configured local Diffusers default (`ABSTRACTCORE_VISION_MODEL_ID` / `ABSTRACTVISION_DIFFUSERS_MODEL_ID` / `ABSTRACTVISION_MODEL_ID`)
  - If `model` is `diffusers/<huggingface-repo>`, uses that explicit local Diffusers model
  - If `model` is `sdcpp/default`, uses the configured stable-diffusion.cpp model
  - If `model` is `openai-compatible/<model>` and an upstream base URL is configured, uses the image proxy
  - If `model` is `openai/gpt-image-1`, or the route is `/openai/v1/images/generations`, uses the OpenAI Images API with `OPENAI_API_KEY`

- **Proxy**: set `ABSTRACTCORE_VISION_BACKEND=openai_compatible_proxy` (proxy to an upstream OpenAI-compatible image server)
  - `OPENAI_BASE_URL` (required) — base URL (include `/v1`)
  - `OPENAI_API_KEY` (optional)
  - `ABSTRACTCORE_VISION_UPSTREAM_MODEL_ID` / `ABSTRACTVISION_MODEL_ID` (optional)
  - `ABSTRACTCORE_VISION_UPSTREAM_VIDEOS_GENERATIONS_PATH` / `ABSTRACTCORE_VISION_TEXT_TO_VIDEO_PATH` (optional, default `/videos/generations`)
  - `ABSTRACTCORE_VISION_UPSTREAM_VIDEOS_EDITS_PATH` / `ABSTRACTCORE_VISION_IMAGE_TO_VIDEO_PATH` (optional, default `/videos/edits`)
  - Install: `pip install "abstractcore[server]"`

- **Local Diffusers**: set `ABSTRACTCORE_VISION_BACKEND=diffusers`
  - `ABSTRACTCORE_VISION_MODEL_ID` / `ABSTRACTVISION_DIFFUSERS_MODEL_ID` / `ABSTRACTVISION_MODEL_ID` (required for `diffusers/default`) — local model id/path (Diffusers)
  - `ABSTRACTCORE_VISION_DEVICE` / `ABSTRACTVISION_DIFFUSERS_DEVICE` (optional, default `auto`; chooses `cuda`/`mps` when available, else `cpu`)
  - `ABSTRACTCORE_VISION_TORCH_DTYPE` / `ABSTRACTVISION_DIFFUSERS_TORCH_DTYPE` (optional, e.g. `float16`; for very large models you typically need `float16` or you may run out of memory)
  - `ABSTRACTCORE_VISION_ALLOW_DOWNLOAD` / `ABSTRACTVISION_DIFFUSERS_ALLOW_DOWNLOAD` (optional, default false/cache-only)
  - Note: if you set `ABSTRACTCORE_VISION_DEVICE=mps` or `cuda`, your PyTorch must actually support it (`torch.backends.mps.is_available()` / `torch.cuda.is_available()`).
  - Install: `pip install "abstractcore[server,vision]"`

- **Local stable-diffusion.cpp**: set `ABSTRACTCORE_VISION_BACKEND=sdcpp`
  - Recommended (pip-only): `pip install "abstractcore[server,vision]"`
  - Alternative (external executable): install `sd-cli`: https://github.com/leejet/stable-diffusion.cpp/releases
  - `ABSTRACTCORE_VISION_SDCPP_BIN` (optional, default `sd-cli`)
  - Configure either:
    - **Full model**: `ABSTRACTCORE_VISION_SDCPP_MODEL`
    - **Component mode**: `ABSTRACTCORE_VISION_SDCPP_DIFFUSION_MODEL` (and optional components like `ABSTRACTCORE_VISION_SDCPP_VAE`, `ABSTRACTCORE_VISION_SDCPP_LLM`, ...)
  - `ABSTRACTCORE_VISION_SDCPP_EXTRA_ARGS` (optional) — extra `sd-cli` flags (e.g. `--diffusion-fa --sampling-method euler --flow-shift 3`)

**Text-to-image (JSON)**:
```bash
# Required for diffusers/default:
export ABSTRACTCORE_VISION_MODEL_ID="org/image-model"

curl http://localhost:8000/v1/images/generations \\
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \\
  -H "Content-Type: application/json" \\
  -d '{"model":"diffusers/default","prompt":"a red fox in snow","width":512,"height":512,"steps":20,"response_format":"b64_json"}'
```

**Image edit (multipart/form-data)**:
```bash
# Required for diffusers/default:
export ABSTRACTCORE_VISION_MODEL_ID="org/image-model"

curl http://localhost:8000/v1/images/edits \\
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \\
  -F "model=diffusers/default" \\
  -F "prompt=make it watercolor" \\
  -F "image=@./input.png" \\
  -F "reference_images=@./style-reference.png"
```

**Text-to-video (JSON, async job with progress polling)**:
```bash
curl http://localhost:8000/v1/vision/jobs/videos/generations \\
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \\
  -H "Content-Type: application/json" \\
  -d '{"provider":"mlx-gen","model":"AbstractFramework/wan2.2-t2v-a14b-diffusers-8bit","prompt":"A slow camera move through a luminous data center.","width":432,"height":240,"fps":24,"num_frames":41,"steps":20,"guidance_scale":4.0,"guidance_2":3.0,"extra":{"max_sequence_length":256}}'
```

Poll `GET /v1/vision/jobs/{job_id}`. Image and video jobs expose
`progress.last_event` when the selected backend reports a normalized progress
event. For MLX-Gen, `progress` is denoise-step progress; video frame context is
exposed separately as `frame`, `total_frames`, and `frame_progress`.

**Image-to-video (multipart/form-data)**:
```bash
curl http://localhost:8000/v1/videos/edits \\
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \\
  -F "provider=mlx-gen" \\
  -F "model=AbstractFramework/wan2.2-i2v-a14b-diffusers-8bit" \\
  -F "prompt=Slow camera push-in." \\
  -F "image=@./first-frame.png" \\
  -F "width=432" \\
  -F "height=240" \\
  -F "num_frames=41" \\
  -F "fps=24" \\
  -F "steps=20" \\
  -F "guidance_scale=3.5" \\
  -F "guidance_2=3.5" \\
  -F 'extra_json={"max_sequence_length":256}'
```

Notes:
- The server returns `b64_json` outputs, matching the OpenAI media API shape.
- OpenAI-compatible image proxying is built into `abstractcore[server]`.
- Wan A14B video models use `guidance_2` as a typed second-stage/low-noise
  guidance control. Keep backend-only values such as `max_sequence_length` in
  `extra` / `extra_json`.
- Strict OpenAI-compatible upstreams receive only OpenAI-compatible top-level
  fields by default. Put backend-specific knobs such as `seed`, `steps`,
  `guidance_scale`, or `negative_prompt` in `extra` / `extra_json` only when
  your custom upstream supports them.
- Local Diffusers/sdcpp/MLX-Gen generation delegates to AbstractVision; install it in the same env as the server with `pip install "abstractcore[server,vision]"` and add the backend runtime extra you need, such as `pip install "abstractvision[mlx-gen]"` for MLX-Gen. Aggregate local profiles such as `abstractcore[all-apple]` include the MLX-Gen path. Prefer `python -m uvicorn ...`.
- Local Diffusers is cache-only by default, matching AbstractVision. Pre-download model weights or opt in with `ABSTRACTCORE_VISION_ALLOW_DOWNLOAD=1`.
- MLX-Gen quantized prepared models are selected by their exact published repo id (for example `AbstractFramework/qwen-image-2512-4bit`, `AbstractFramework/seedvr2-3b-8bit`, `AbstractFramework/seedvr2-7b-4bit`, `AbstractFramework/wan2.2-t2v-a14b-diffusers-8bit`, or `AbstractFramework/wan2.2-i2v-a14b-diffusers-8bit`). For SeedVR2 official/source-weight upscaling only, the `quantize` form field can request runtime quantization (`3`, `4`, `5`, `6`, or `8`). Prepared q4/q8 package ids do not need `quantize`. Use `ABSTRACTCORE_VISION_MODEL_DIR` or `ABSTRACTVISION_MODEL_DIR` when prepared folders live outside the Hugging Face cache.

---

### 6. Document Processing

**Endpoint**: `POST /v1/chat/completions` (with document attachments)

**Purpose**: Process PDF, CSV, Excel, JSON, XML, and text documents.

**Supported Document Types**:
- **Documents**: PDF, DOCX, XLSX, PPTX
- **Data**: CSV, TSV, JSON, XML
- **Text**: TXT, MD, HTML
- Size limit: 10MB per file, 32MB total

**Request with CSV File**:
```json
{
  "model": "lmstudio/qwen3-next-80b",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "Analyze this CSV and calculate total sales"},
      {
        "type": "image_url",
        "image_url": {
          "url": "data:text/csv;base64,RGF0ZSxQcm9kdWN0LFNhbGVzCjIwMjQtMDEtMDEsUHJvZHVjdCBBLDEwMDAwCjIwMjQtMDEtMDIsUHJvZHVjdCBCLDE1MDAwCjIwMjQtMDEtMDMsUHJvZHVjdCBDLDI1MDAw"
        }
      }
    ]
  }]
}
```

**Request with PDF**:
```json
{
  "model": "openai/gpt-4o",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "Summarize this PDF report"},
      {
        "type": "image_url",
        "image_url": {
          "url": "data:application/pdf;base64,JVBERi0xLjQKJdPr6eEK..."
        }
      }
    ]
  }]
}
```

**Python with CSV**:
```python
import requests
import base64

# Read and encode CSV
with open("sales.csv", "rb") as f:
    csv_base64 = base64.b64encode(f.read()).decode()

response = requests.post(
    "http://localhost:8000/v1/chat/completions",
    json={
        "model": "lmstudio/qwen3-next-80b",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": "Calculate total sales from this data"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:text/csv;base64,{csv_base64}"}
                }
            ]
        }]
    }
)

print(response.json()["choices"][0]["message"]["content"])
```

---

### 7. Embeddings

**Endpoint**: `POST /v1/embeddings`

**Purpose**: Generate text embeddings using embedding models.

Supported providers include local/native providers (`huggingface`, `ollama`)
and OpenAI-compatible remote providers (`lmstudio`, `openai`, `openrouter`,
`portkey`, `openai-compatible`). Anthropic does not expose a native embeddings
API, so use another embeddings provider for RAG/vector workflows.

**Request**:
```json
{
  "model": "huggingface/sentence-transformers/all-MiniLM-L6-v2",
  "input": "The quick brown fox jumps over the lazy dog",
  "encoding_format": "float",
  "dimensions": 0
}
```

**Request (Batch)**:
```json
{
  "model": "ollama/granite-embedding:278m",
  "input": [
    "First text to embed",
    "Second text to embed",
    "Third text to embed"
  ]
}
```

**Response**:
```json
{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [0.123, -0.456, 0.789, ...],
      "index": 0
    }
  ],
  "model": "huggingface/sentence-transformers/all-MiniLM-L6-v2",
  "usage": {
    "prompt_tokens": 12,
    "total_tokens": 12
  }
}
```

**Usage**:
```bash
curl -X POST http://localhost:8000/v1/embeddings \
  -H "Authorization: Bearer $ABSTRACTCORE_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": "The quick brown fox jumps over the lazy dog",
    "dimensions": 256
  }'
```

**Python**:
```python
import requests

response = requests.post(
    "http://localhost:8000/v1/embeddings",
    json={
        "model": "huggingface/sentence-transformers/all-MiniLM-L6-v2",
        "input": "The quick brown fox jumps over the lazy dog"
    }
)

embedding = response.json()["data"][0]["embedding"]
print(f"Embedding dimension: {len(embedding)}")
```

---

### 8. OpenAI Responses API

**Endpoint**: `POST /v1/responses`

**Purpose**: OpenAI Responses API compatible endpoint with file URL support (`input` requests return `object: "response"`; legacy `messages` requests return Chat Completions payloads).

**Request**:
```json
{
  "model": "openai/gpt-4o",
  "input": [
    {
      "role": "user",
      "content": [
        {"type": "input_text", "text": "Analyze this file"},
        {"type": "input_file", "file_url": "https://example.com/data.csv"}
      ]
    }
  ],
  "instructions": "You are a helpful assistant.",
  "tools": [{"type": "web_search", "external_web_access": true}],
  "tool_choice": "auto",
  "max_output_tokens": 512,
  "stream": false
}
```

**Note**: This endpoint accepts both OpenAI Responses format (with `input`) and legacy format (with `messages`). Format is auto-detected. AbstractCore does not execute tools server-side; tools are only transported to the model and emitted back as tool calls for the host to run.

---

### 9. Provider-Specific Endpoints

**Endpoint**: `POST /{provider}/v1/chat/completions`

**Purpose**: Route requests to specific provider without provider prefix in model name.

**Example**:
```bash
# Standard endpoint with provider prefix
curl -X POST http://localhost:8000/v1/chat/completions \
  -d '{"model": "openai/gpt-4o-mini", "messages": [...]}'

# Provider-specific endpoint without prefix
curl -X POST http://localhost:8000/openai/v1/chat/completions \
  -d '{"model": "gpt-4o-mini", "messages": [...]}'
```

**Supported Providers**:
- `/openai/v1/chat/completions`
- `/anthropic/v1/chat/completions`
- `/ollama/v1/chat/completions`
- `/lmstudio/v1/chat/completions`
- `/huggingface/v1/chat/completions`
- `/mlx/v1/chat/completions`

---

## Media Processing Features

### Automatic Format Detection

The server automatically detects and processes:

| MIME Type | Extensions | Processing |
|-----------|------------|------------|
| `image/*` | png, jpg, gif, webp, bmp, tiff | Vision models |
| `application/pdf` | pdf | Text extraction |
| `text/csv` | csv | Table rendering |
| `application/vnd.ms-excel` | xls, xlsx | Table rendering |
| `application/json` | json | JSON formatting |
| `text/plain` | txt, md | Text rendering |
| `text/html` | html | Text extraction |

### Data URL Support

Supports both base64 data URLs and HTTP(S) URLs:

```json
{
  "type": "image_url",
  "image_url": {
    "url": "data:image/png;base64,iVBORw0KGgo..."
  }
}
```

```json
{
  "type": "image_url",
  "image_url": {
    "url": "https://example.com/image.png"
  }
}
```

### Size Limits

- Per file: 10MB
- Total per request: 32MB
- Automatic validation and error messages

---

## Tool Call Syntax Support

The server includes advanced tool call syntax conversion:

### Supported Formats

| Format | Description | Detection |
|--------|-------------|-----------|
| `openai` | OpenAI function calling | Default for OpenAI models |
| `codex` | Codex-style function calls | Auto-detect from User-Agent |
| `qwen3` | Qwen3 tool syntax | Auto-detect from model name |
| `llama3` | LLaMA3 tool syntax | Auto-detect from model name |
| `passthrough` | No conversion | Explicit opt-in |
| `auto` | Automatic detection | Recommended (default) |

### Agent Format Parameter

Control tool syntax with `agent_format` parameter:

```json
{
  "model": "anthropic/claude-3-opus-20240229",
  "messages": [...],
  "tools": [...],
  "agent_format": "auto"
}
```

### Auto-Detection Rules

1. **User-Agent Detection**: Codex clients get Codex format
2. **Model Name Detection**: qwen3 → Qwen3, llama3 → LLaMA3
3. **Provider Default**: OpenAI models → OpenAI format
4. **Fallback**: OpenAI format (most compatible)

---

## Deployment

### Local Development

**Start Server**:
```bash
# Standard mode
python -m abstractcore.server

# Debug mode (verbose logging)
python -m abstractcore.server --debug

# Custom host/port
python -m abstractcore.server --host 0.0.0.0 --port 8080

# All options
python -m abstractcore.server --host 0.0.0.0 --port 8080 --debug
```

**Environment Variables**:
```bash
# Provider API keys
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."

# Server auth token
export ABSTRACTCORE_AUTH_TOKEN="acore-server-secret"

# Debug mode
export ABSTRACTCORE_DEBUG="true"

# Start server
python -m abstractcore.server
```

### Production Deployment

**Using Uvicorn Directly**:
```bash
uvicorn abstractcore.server.app:app --host 0.0.0.0 --port 8000 --workers 4
```

**Using Gunicorn + Uvicorn Workers**:
```bash
gunicorn abstractcore.server.app:app \
  --worker-class uvicorn.workers.UvicornWorker \
  --workers 4 \
  --bind 0.0.0.0:8000
```

**Docker Deployment**:
```bash
docker run -p 8000:8000 \
  -e ABSTRACTCORE_AUTH_TOKEN="acore-server-secret" \
  -e OPENAI_API_KEY="sk-..." \
  -e OPENROUTER_API_KEY="sk-or-..." \
  ghcr.io/lpalbou/abstractcore-server:2.13.11
```

The release image is built from PyPI with
`abstractcore[server,remote,media,tokens,compression]==<version>`. It is a
remote/server gateway image and intentionally does not bundle local model
runtimes, local Diffusers/sdcpp vision backends, `sentence-transformers`, or
AbstractVoice/AbstractVision plugin entry points. Remote image/audio
OpenAI-compatible endpoint routes still work. Use a custom image with
`abstractcore[server,remote,media,tokens,compression,voice,vision]` when you
want plugin-backed media catalogs or plugin default routes.

### Nginx Reverse Proxy

```nginx
server {
    listen 80;
    server_name api.example.com;

    location / {
        proxy_pass http://localhost:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Streaming support
        proxy_buffering off;
        proxy_cache off;
    }
}
```

---

## Configuration

### API Keys

The server automatically reads API keys from environment variables:

```bash
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export ABSTRACTCORE_AUTH_TOKEN="acore-server-secret"
```

Or use AbstractCore's configuration system:
```bash
abstractcore --set-api-key openai sk-...
abstractcore --set-api-key anthropic sk-ant-...
```

### Logging

Configure logging via environment or code:

**Environment**:
```bash
export ABSTRACTCORE_DEBUG="true"
python -m abstractcore.server
```

**Code** (`abstractcore/server/app.py`):
```python
from abstractcore.utils.structured_logging import configure_logging
import logging

configure_logging(
    console_level=logging.INFO,
    file_level=logging.DEBUG,
    log_dir="logs",
    console_json=False,
    file_json=True
)
```

### CORS

CORS is enabled by default for all origins. To restrict:

```python
# In app.py
app.add_middleware(
    CORSMiddleware,
    allow_origins=["https://example.com"],  # Specific origins
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)
```

---

## Error Handling

### HTTP Status Codes

| Code | Meaning | Example |
|------|---------|---------|
| 200 | Success | Request completed successfully |
| 400 | Bad Request | Invalid JSON or missing fields |
| 422 | Validation Error | Invalid parameter values |
| 404 | Not Found | Unknown endpoint or model |
| 500 | Server Error | Internal processing failure |
| 503 | Service Unavailable | Provider unavailable |

### Error Response Format

```json
{
  "error": {
    "message": "Request validation failed",
    "type": "validation_error",
    "details": [
      {
        "field": "model",
        "message": "field required",
        "type": "value_error.missing"
      }
    ]
  }
}
```

### Common Errors

**Invalid Model Format**:
```json
{
  "error": {
    "message": "Invalid model format. Use 'provider/model' format (e.g., 'openai/gpt-4o-mini')",
    "type": "validation_error"
  }
}
```

**Provider Not Available**:
```json
{
  "error": {
    "message": "Provider 'xyz' not found. Available providers: openai, anthropic, ollama, ...",
    "type": "provider_error"
  }
}
```

**Media Processing Error**:
```json
{
  "error": {
    "message": "File size exceeds maximum limit of 10MB",
    "type": "media_processing_error"
  }
}
```

---

## Client Integration

### Python (requests)

```python
import requests

class AbstractCoreClient:
    def __init__(self, base_url="http://localhost:8000"):
        self.base_url = base_url

    def chat(self, model, messages, **kwargs):
        response = requests.post(
            f"{self.base_url}/v1/chat/completions",
            json={"model": model, "messages": messages, **kwargs}
        )
        response.raise_for_status()
        return response.json()

    def embed(self, model, input_text):
        response = requests.post(
            f"{self.base_url}/v1/embeddings",
            json={"model": model, "input": input_text}
        )
        response.raise_for_status()
        return response.json()

# Usage
client = AbstractCoreClient()
result = client.chat(
    "openai/gpt-4o-mini",
    [{"role": "user", "content": "Hello!"}]
)
```

### Python (OpenAI SDK)

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key=os.environ["ABSTRACTCORE_AUTH_TOKEN"]
)

response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

### JavaScript/TypeScript

```typescript
interface ChatMessage {
  role: 'system' | 'user' | 'assistant';
  content: string;
}

async function chat(model: string, messages: ChatMessage[]) {
  const response = await fetch('http://localhost:8000/v1/chat/completions', {
    method: 'POST',
    headers: {'Content-Type': 'application/json'},
    body: JSON.stringify({model, messages})
  });

  if (!response.ok) {
    throw new Error(`HTTP ${response.status}: ${await response.text()}`);
  }

  return response.json();
}

// Usage
const result = await chat('openai/gpt-4o-mini', [
  {role: 'user', content: 'Hello!'}
]);
console.log(result.choices[0].message.content);
```

### cURL

```bash
# Simple chat
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'

# Streaming
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -N \
  -d '{"model":"ollama/llama3.2:3b","messages":[{"role":"user","content":"Hello!"}],"stream":true}'

# Embeddings
curl -X POST http://localhost:8000/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model":"huggingface/sentence-transformers/all-MiniLM-L6-v2","input":"Hello world"}'
```

---

## Best Practices

### 1. Use Provider/Model Format

Always specify provider prefix:
```json
{"model": "openai/gpt-4o-mini"}  // Correct
{"model": "gpt-4o-mini"}         // May fail
```

### 2. Handle Streaming Properly

For streaming responses, process server-sent events:
```python
import requests

response = requests.post(
    "http://localhost:8000/v1/chat/completions",
    json={"model": "ollama/llama3.2:3b", "messages": [...], "stream": True},
    stream=True
)

for line in response.iter_lines():
    if line:
        line = line.decode('utf-8')
        if line.startswith('data: '):
            data = line[6:]
            if data == '[DONE]':
                break
            import json
            chunk = json.loads(data)
            print(chunk['choices'][0]['delta'].get('content', ''), end='')
```

### 3. Optimize Media Attachments

- Compress images before sending
- Use appropriate image resolution (don't send 10MP when 1MP suffices)
- Consider HTTP URLs instead of base64 for large files

### 4. Implement Retry Logic

```python
import time
import requests

def chat_with_retry(model, messages, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = requests.post(
                "http://localhost:8000/v1/chat/completions",
                json={"model": model, "messages": messages},
                timeout=60
            )
            response.raise_for_status()
            return response.json()
        except requests.exceptions.RequestException as e:
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)  # Exponential backoff
                continue
            raise
```

### 5. Monitor Token Usage

Track usage for cost management:
```python
result = client.chat(model, messages)
usage = result["usage"]
print(f"Tokens: {usage['total_tokens']} (prompt: {usage['prompt_tokens']}, completion: {usage['completion_tokens']})")
```

---

## Performance Considerations

### Concurrent Requests

The server handles concurrent requests efficiently:

```python
from concurrent.futures import ThreadPoolExecutor

def process_request(text):
    return client.chat("openai/gpt-4o-mini", [{"role": "user", "content": text}])

texts = ["Query 1", "Query 2", "Query 3", ...]

with ThreadPoolExecutor(max_workers=10) as executor:
    results = list(executor.map(process_request, texts))
```

### Streaming for Long Responses

Use streaming for better UX with long responses:
```python
response = client.chat(
    "openai/gpt-4o-mini",
    messages=[...],
    stream=True,
    max_tokens=4096
)
```

### Batch Embeddings

Process multiple texts in one request:
```python
client.embed(
    "huggingface/sentence-transformers/all-MiniLM-L6-v2",
    ["Text 1", "Text 2", "Text 3", ...]
)
```

---

## Monitoring and Observability

### Structured Logging

All requests are logged with structured data:

```json
{
  "timestamp": "2025-11-06T12:00:00Z",
  "level": "INFO",
  "message": "HTTP Request",
  "method": "POST",
  "url": "/v1/chat/completions",
  "status_code": 200,
  "process_time_ms": 1234.56,
  "model": "openai/gpt-4o-mini",
  "provider": "openai",
  "usage": {"total_tokens": 150}
}
```

### Debug Mode

Enable comprehensive debugging:
```bash
python -m abstractcore.server --debug
```

Debug mode logs:
- Request headers and body
- Provider API calls
- Media processing steps
- Tool call conversions
- Full stack traces

### Health Monitoring

Monitor server health:
```bash
# Check health
curl http://localhost:8000/health

# Expected response
{"status":"healthy","version":"2.5.2","timestamp":"..."}
```

---

## Troubleshooting

### Server Won't Start

**Problem**: Import errors or port conflicts.

**Solution**:
```bash
# Check dependencies
pip install -r requirements.txt

# Check port availability
lsof -i :8000

# Use different port
python -m abstractcore.server --port 8080
```

### Model Not Found

**Problem**: Model not listed in `/v1/models`.

**Solution**:
1. Check provider is installed: `pip list | grep openai`
2. Verify API key is set: `echo $OPENAI_API_KEY`
3. Check provider status: `curl http://localhost:8000/providers`

### Streaming Not Working

**Problem**: Streaming responses not received.

**Solution**:
- Disable proxy buffering (Nginx: `proxy_buffering off;`)
- Use `stream=True` in requests library
- Process lines individually, not full response

### Media Processing Fails

**Problem**: Images/documents not processed.

**Solution**:
1. Check file size (max 10MB per file)
2. Verify base64 encoding is correct
3. Check MIME type in data URL
4. Ensure model supports vision/document processing

---

## Future Enhancements

Planned improvements:
- WebSocket support for real-time streaming
- GraphQL API alongside REST
- Built-in rate limiting and quotas
- Caching layer for repeated requests
- Metrics endpoint (Prometheus format)
- OpenTelemetry tracing
- Multi-tenancy support
- Request replay and debugging UI

---

For detailed implementation, see `/Users/albou/projects/abstractcore/abstractcore/server/app.py` (2000+ lines).

## Related Modules

**Direct dependencies**:
- [`core/`](../core/README.md) - LLM factory and generation
- [`providers/`](../providers/README.md) - Provider registry and health checks
- [`config/`](../config/README.md) - Server configuration management
- [`processing/`](../processing/README.md) - High-level processors
- [`media/`](../media/README.md) - Media upload and processing
- [`structured/`](../structured/README.md) - API response schemas
- [`exceptions/`](../exceptions/README.md) - Error response formatting
- [`utils/`](../utils/README.md) - Logging, metrics

**Exposes**:
- [`architectures/`](../architectures/README.md) - Model capabilities via API
- [`assets/`](../assets/README.md) - Provider and model listings
- [`tools/`](../tools/README.md) - Tool execution endpoints
- [`embeddings/`](../embeddings/README.md) - Embedding generation endpoints

**Related systems**:
- [`events/`](../events/README.md) - Server event emission
