高性能AI网关

Installation

```toml

Build/test uses too much CPU or memory

• Use API-only defaults first: cargo test --lib --tests --no-default-features --features "lite"
• Limit local parallelism when needed: CARGO_BUILD_JOBS=4 cargo test --lib --tests --no-default-features --features "lite" -- --test-threads=4
• Avoid --all-features unless you are doing release/nightly validation

Quick Start (5 Minutes, API-Only Recommended)

Most users use this project as a unified API library, not as a gateway server. Start with API-only mode first.

[dependencies]
litellm-rs = { version = "0.5", default-features = false, features = ["lite"] }

For crate users, no make is required.

Usage

Examples

Environment Variables

```bash

Optional

LITELLM_VERBOSE=true # Enable verbose logging ```

As a Library (API Integration)

use litellm_rs::{completion, user_message, system_message};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let response = completion(
        "gpt-4",
        vec![
            system_message("You are a helpful assistant."),
            user_message("Hello!"),
        ],
        None,
    ).await?;

    println!("{}", response.choices[0].message.content.as_ref().unwrap());
    Ok(())
}

API-only - lightweight, no actix-web/argon2/aes-gcm/clap

[dependencies] litellm-rs = { version = "0.5", default-features = false }

API-only with metrics

[dependencies] litellm-rs = { version = "0.5", default-features = false, features = ["lite"] }

Provider API Keys

OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... GOOGLE_API_KEY=... AZURE_OPENAI_API_KEY=... AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_REGION=us-east-1 GROQ_API_KEY=... DEEPSEEK_API_KEY=... MOONSHOT_API_KEY=... ZHIPU_API_KEY=... MINIMAX_API_KEY=...

I only need provider API aggregation, not gateway

• Prefer default-features = false with features = ["lite"]
• Use gateway runtime commands only when you need HTTP server/auth/storage middleware

Gateway modules in library context (not standalone gateway binary runtime)

[dependencies] litellm-rs = { version = "0.5", default-features = false, features = ["gateway"] } ```

Experimental / module-only

The following modules exist under src/core/providers/ (gated on providers-extra or providers-extended) but are not wired into the unified Provider enum or the factory today. They compile but cannot be selected through create_provider/from_config_async. Treat them as experimental scaffolding subject to change:

ai21, baseten, clarifai, codestral, cohere, custom_api, databricks, datarobot, deepgram, deepl, elevenlabs, empower, exa_ai, firecrawl, gemini, gigachat, google_pse, gradient_ai, huggingface, jina, langgraph, manus, milvus, morph, nlp_cloud, oci, ollama, petals, pg_vector, predibase, ragflow, recraft, runwayml, sagemaker, sap_ai, searxng, snowflake, spark, stability, tavily, topaz, triton, vercel_ai, voyage, watsonx

For self-hosted or unlisted OpenAI-compatible endpoints, prefer the generic openai_compatible provider type instead.

Troubleshooting