# Local & Self-Hosted Providers

This page covers providers that run on your machine, inside your network, or behind a self-hosted inference endpoint that LibreFang can reach directly.

## Included Providers

- Ollama
- vLLM
- LM Studio

---

## Ollama

| | |
|---|---|
| **Display Name** | Ollama |
| **Driver** | OpenAI-compatible |
| **Env Var** | `OLLAMA_API_KEY` (not required) |
| **Base URL** | `http://localhost:11434/v1` |
| **Key Required** | **No** |
| **Free Tier** | Free (local) |
| **Auth** | None (local) |
| **Models** | 3 builtin + auto-discovered |

**Available Models (builtin):**
- `llama3.2` (Local)
- `mistral:latest` (Local)
- `phi3` (Local)

**Setup:**
1. Install Ollama from [ollama.com](https://ollama.com)
2. Pull a model: `ollama pull llama3.2`
3. Start the server: `ollama serve`
4. No env var needed -- Ollama is always available

**Notes:** LibreFang auto-discovers models from a running Ollama instance and merges them into the catalog with `Local` tier and zero cost. Any model you pull becomes usable immediately.

---

## vLLM

| | |
|---|---|
| **Display Name** | vLLM |
| **Driver** | OpenAI-compatible |
| **Env Var** | `VLLM_API_KEY` (not required) |
| **Base URL** | `http://localhost:8000/v1` |
| **Key Required** | **No** |
| **Free Tier** | Free (self-hosted) |
| **Auth** | None (local) |
| **Models** | 1 builtin + auto-discovered |

**Available Models (builtin):**
- `vllm-local` (Local)

**Setup:**
1. Install vLLM: `pip install vllm`
2. Start the server: `python -m vllm.entrypoints.openai.api_server --model <model-name>`
3. No env var needed

---

## LM Studio

| | |
|---|---|
| **Display Name** | LM Studio |
| **Driver** | OpenAI-compatible |
| **Env Var** | `LMSTUDIO_API_KEY` (not required) |
| **Base URL** | `http://localhost:1234/v1` |
| **Key Required** | **No** |
| **Free Tier** | Free (local) |
| **Auth** | None (local) |
| **Models** | 1 builtin + auto-discovered |

**Available Models (builtin):**
- `lmstudio-local` (Local)

**Setup:**
1. Download LM Studio from [lmstudio.ai](https://lmstudio.ai)
2. Download a model from the built-in model browser
3. Start the local server from the "Local Server" tab
4. No env var needed

---

## Embedding auto-detection

When the embeddings driver is set to `provider = "auto"`, LibreFang walks a priority list to pick the first reachable embeddings backend. Cloud API keys win first; local servers come after. The current order (verified against `librefang-runtime::embedding::detect_embedding_provider`):

1. `OPENAI_API_KEY` → `"openai"`
2. `OPENROUTER_API_KEY` → `"openrouter"`
3. `MISTRAL_API_KEY` → `"mistral"`
4. `TOGETHER_API_KEY` → `"together"`
5. `FIREWORKS_API_KEY` → `"fireworks"`
6. `COHERE_API_KEY` → `"cohere"`
7. `OLLAMA_HOST` (local)
8. `VLLM_BASE_URL` (local)
9. `LMSTUDIO_BASE_URL` (local)

Setting any of `OLLAMA_HOST` / `VLLM_BASE_URL` / `LMSTUDIO_BASE_URL` to a reachable URL is enough — no API key needed for the local entries. The same priority is applied by `create_embedding_driver`, so what `auto` picks at startup matches what an explicit `provider = "vllm"` would build.

`GROQ_API_KEY` is deliberately **not** in the ladder — Groq exposes no `/v1/embeddings` endpoint, so auto-picking it would produce silent 404s at the first real call. Likewise, providers without a wired embedding driver (`VOYAGE`, `JINA`, `MIXEDBREAD`, `BEDROCK` for auto-pick — Bedrock works when explicitly configured) are not auto-detected even if their chat keys are set.

---

## Capability detection (Ollama metadata pipeline)

For Ollama, LibreFang probes `/api/tags` at startup and on the periodic provider refresh, then derives each model's capabilities from the metadata it returns:

| Capability | How it's inferred |
|---|---|
| `vision` | Model card declares an image-input modality, or the family is in the known-vision list (`llava`, `bakllava`, `clip`, …) |
| `embedding` | Model card declares an embedding-only architecture (`nomic-embed-text`, `bge-m3`, `mxbai-embed-large`, …) |
| `tools` | Model card declares tool-use support (newer Llama / Qwen / Mistral instruct families) |
| `parameter_size` | Lifted verbatim from the Ollama tag (e.g. `7.6B`, `14.8B`) |
| `quantization_level` | Lifted verbatim (e.g. `Q4_K_M`, `F16`) |

The dashboard's "Models" tab and the model-switcher dropdown show these badges per model, and they feed into the `model_catalog`'s `Modality` enum (`Text` / `Image` / `Audio`) — a caller asking for image-generation models filters cleanly without a hard-coded list.

vLLM and LM Studio receive the same probe via their OpenAI-compat `/models` endpoint, but the Ollama path populates the most fields because the upstream API exposes the richest data.

### Reprobe cadence

Local providers are probed at startup and **re-probed every 60 seconds** in the background — so a model you `ollama pull` while the daemon is running shows up in the catalog without a daemon restart, and a stopped local server flips to `auth_status: "local_offline"` automatically. The dashboard's **"Test connection"** button on a provider card forces an immediate re-probe + cache refresh so you don't have to wait for the next periodic tick.
