本地LLM路由器

Install

Quick start

```bash mkdir -p ~/.config/routectl routectl config example > ~/.config/routectl/config.toml

Configuration

Minimal example:

[server]
host = "127.0.0.1"
port = 8787

[providers.deepseek]
kind = "openai-compat"
base_url = "https://api.deepseek.com/v1"
api_key_ref = "env://DEEPSEEK_API_KEY"
reasoning_dialect = "deepseek"

[providers.anthropic]
kind = "anthropic-api"
api_key_ref = "env://ANTHROPIC_API_KEY"

[providers.bedrock]
kind = "bedrock"
region = "us-east-1"
creds = { kind = "bearer-key", key_ref = "env://AWS_BEARER_TOKEN_BEDROCK" }

[models.fast]
provider = "deepseek"
upstream = "deepseek-chat"

[models.heavy-bedrock]
provider = "bedrock"
upstream = "us.anthropic.claude-opus-4-20250514-v1:0"
supports_adaptive_thinking = true
effort_levels = ["low", "medium", "high"]

[models.heavy-anthropic]
provider = "anthropic"
upstream = "claude-opus-4-20250514"
supports_adaptive_thinking = true
effort_levels = ["low", "medium", "high"]

[aliases]
fast  = "fast"
heavy = ["heavy-bedrock", "heavy-anthropic"]   # fallback chain
"claude-opus-*" = "heavy"                       # suffix-glob routing
default = "fast"

See examples/config.toml for the full surface, including the global [retry] table, per-provider RPM limits, and circuit breakers. v0.6 collapsed per-alias retry overrides into the workspace-global [retry] -- per-error-class caps (retry_on_429, retry_on_5xx, retry_on_network) cover the same operator knobs.

Raw access key + secret key (+ optional session token).

[providers.bedrock-static.creds] kind = "static" access_key_ref = "env://AWS_ACCESS_KEY_ID" secret_key_ref = "env://AWS_SECRET_ACCESS_KEY" session_token_ref = "env://AWS_SESSION_TOKEN" # optional

Named profile. SSO sessions auto-refresh via aws-config.

creds = { kind = "profile", name = "my-bedrock-profile" }

Standard AWS chain: env -> profile -> SSO -> IRSA -> IMDS.

creds = { kind = "default-chain" }

`api_shape = "invoke"` (default) sends the per-vendor body shape -- Anthropic Messages JSON for Claude. `api_shape = "converse"` uses the AWS vendor-neutral envelope; both are wired and live-tested for Anthropic models. Converse for non-Anthropic vendors (Mistral, Llama, Cohere) is staged.

Bedrock-specific knobs:

- `user_agent` -- per-provider UA override. Required when an IAM policy gates access via the `aws:UserAgent` condition key.
- `anthropic_beta` -- list of beta flags always sent on requests from this provider (operator-asserted floor; bypasses the global allowlist filter below).
- `additional_model_request_fields` -- free-form JSON merged into the request body for vendor-specific knobs.

**`[bedrock]` allowlists (optional, recommended in production).** AWS strict-schema validation 400s any unrecognized `anthropic_beta` flag or top-level body field. routectl ships no built-in default; populate operator-supplied lists in TOML to gate which entries reach AWS:

Empty lists (or omitted [bedrock] section) = pass-through (no filter applied) -- the discovery default. Run with ROUTECTL_LOG=routectl_providers::bedrock=trace to capture sent flags/fields, then populate the lists. See examples/bedrock.toml for the empirical 2026-05-12 baseline.

BedrockCreds redacts secret material in Debug output (no leaks via panic messages or tracing events).

Live integration matrix (opt-in; skips per-provider when its env key is absent).

export OPENROUTER_API_KEY=... export OPENCODE_GO_API_KEY=... export NIM_API_KEY=... export AWS_BEARER_TOKEN_BEDROCK=... export AWS_REGION=us-east-1

cargo test -p routectl-cli --features live-integration --release \ --test live_matrix -- --nocapture --test-threads=1 ```

See docs/TESTED_MODELS.md for the verified model matrix.

Secret references

Routectl resolves credentials through one of four URI schemes per provider. There is no auto-discovery; you choose the source explicitly.

`anthropic-api`

Hits https://api.anthropic.com/v1/messages. Two auth modes via auth_kind:

• api-key (default) -- x-api-key: <key> header.
• oauth-bearer -- Authorization: Bearer <token> header. Pair with api_key_ref = "oauth://anthropic" for routectl-managed login (see "Managed OAuth login" above).

Beta gates (anthropic-beta flags for prompt caching, 1M context, extended thinking) are independent of auth_kind. Declare them via header_extras:

[providers.anthropic]
header_extras = { "anthropic-beta" = "context-1m-2025-08-07,prompt-caching-2024-07-31" }

header_extras cannot override auth-bearing headers (authorization, x-api-key, host); collisions are dropped with a tracing::warn!.

Provider-level knobs relevant to claude-code-as-a-gateway use:

• forward_client_headers: Vec<String> -- allowlist of incoming client headers (typically x-claude-code-session-id, x-claude-code-agent-id, x-claude-code-parent-agent-id) that pass through to the upstream. Defaults to empty (drop everything).
• context_management = true -- routectl emulates Anthropic's context-management-2025-06-27 beta server-side for upstreams that demand thinking-block echoback but do not implement the beta natively (e.g. DeepSeek /anthropic). Bounded LRU + TTL cache; strips the beta header and body field on egress. See docs/PROVIDER-QUIRKS.md for when to flip it.

Short-term Bedrock API key from the AWS console. Skips SigV4 entirely

Unit + integration tests, no network.

cargo test --workspace --release