可用事为常用的常用器

Install dependencies

mix deps.get

Installation

Igniter Installation (Recommended)

The fastest way to get started is with Igniter:

mix igniter.install req_llm

Manual Installation

Add req_llm to your list of dependencies in mix.exs:

def deps do
  [
    {:req_llm, "~> 1.6"}
  ]
end

Then run:

mix deps.get

Quick Start

```elixir

Access usage metadata after streaming

usage = ReqLLM.StreamResponse.usage(response) ```

Usage Cost Tracking

Every response includes detailed usage and best-effort cost information calculated from normalized provider usage data plus model pricing metadata:

```elixir {:ok, response} = ReqLLM.generate_text("anthropic:claude-haiku-4-5", "Hello")

response.usage #=> %{

Tool & Image Usage

When using web search or generating images, additional usage metadata is available:

```elixir

Web search usage (Anthropic, OpenAI, xAI, Google)

{:ok, response} = ReqLLM.generate_text(model, prompt, provider_options: [web_search: %{max_uses: 5}])

response.usage.tool_usage #=> %{web_search: %{count: 2, unit: "call"}}

response.usage.cost #=> %{tokens: 0.001, tools: 0.02, images: 0.0, total: 0.021}

Image generation usage

{:ok, response} = ReqLLM.generate_image("openai:gpt-image-1.5", prompt)

response.usage.image_usage #=> %{generated: %{count: 1, size_class: "1024x1024"}}

A native ReqLLM telemetry surface is published for every request, including streaming:

- `[:req_llm, :request, :start | :stop | :exception]` for lifecycle timing, summaries, and usage
- `[:req_llm, :reasoning, :start | :update | :stop]` for standardized thinking and reasoning milestones
- `[:req_llm, :token_usage]` for backwards-compatible token and cost measurements

All events share a `request_id` so you can correlate request lifecycle, reasoning lifecycle, and billing data across providers.

For OpenTelemetry, attach `ReqLLM.OpenTelemetry` once to emit GenAI client spans, optional GenAI metrics, cost attributes, and Langfuse-friendly message capture.

See examples/scripts/usage_cost_search_image.exs and run it from examples/ with mix run scripts/usage_cost_search_image.exs for a multi-provider smoke test that validates search tool and image generation cost metadata. For comprehensive documentation, see the Telemetry Guide and Usage & Billing Guide.

StreamResponse Usage Patterns

The new StreamResponse provides flexible access patterns:

```elixir

Keys are picked up from .env files or environment variables - see `ReqLLM.Keys`

model = "anthropic:claude-haiku-4-5"

ReqLLM.generate_text!(model, "Hello world") #=> "Hello! How can I assist you today?"

schema = [name: [type: :string, required: true], age: [type: :pos_integer]] person = ReqLLM.generate_object!(model, "Generate a person", schema) #=> %{name: "John Doe", age: 30}

{:ok, image_response} = ReqLLM.generate_image("openai:gpt-image-1.5", "A simple red square") image_bytes = ReqLLM.Response.image_data(image_response) File.write!("red_square.png", image_bytes)

Note: Google image models gemini-2.5-flash-image and gemini-3-pro-image-preview reject :n; specify the image count in the prompt.

{:ok, response} = ReqLLM.generate_text( model, "What's the weather in Paris?", tools: [ ReqLLM.tool( name: "get_weather", description: "Get current weather for a location", parameter_schema: [ location: [type: :string, required: true, doc: "City name"] ], callback: {Weather, :fetch_weather, [:extra, :args]} ) ] )

Streaming Configuration

ReqLLM uses Finch for streaming connections with automatic connection pooling. By default, we use HTTP/1-only pools to avoid a known Finch mixed-protocol ALPN bug with large request bodies:

```elixir

Default configuration (automatic)

config :req_llm, stream_pool_timeout: 120_000, stream_pool_protocols: [:http1], stream_pool_size: 1, stream_pool_count: 8 ```

HTTP/2 Configuration (Advanced)

Important: Due to Finch issue #265, mixed HTTP/1+HTTP/2 ALPN pools may fail when sending request bodies larger than 64KB (large prompts, extensive context windows). This is a bug in Finch's mixed-protocol flow control path, not a limitation of HTTP/2 itself.

If you know all target providers support HTTP/2, you can configure HTTP/2-only pools:

```elixir

HTTP/2-only configuration

config :req_llm, stream_pool_protocols: [:http2], stream_pool_count: 8

**ReqLLM will error with a helpful message if you try to send a large request body with mixed HTTP/1+HTTP/2 pools.** The error will reference this section for configuration guidance.

Streaming responses hold a connection until completion. For high-scale deployments, tune both the Finch pool capacity and the stream checkout timeout:

High-scale configuration

config/runtime.exs

round_robin = Finch.Pool.Strategy.RoundRobin.new()

config :req_llm, stream_pool_timeout: 300_000, stream_pool_protocols: [:http1], stream_pool_size: 1, stream_pool_count: 32, stream_pool_strategy: {Finch.Pool.Strategy.RoundRobin, round_robin}

With the default HTTP/1 transport, concurrent streams per origin are roughly `stream_pool_size * stream_pool_count`. Prefer increasing `stream_pool_count` first when a single pool worker is under pressure; increase `stream_pool_size` when each worker should hold more concurrent HTTP/1 connections. For high worker counts, a round-robin `stream_pool_strategy` spreads stream starts more evenly than Finch's default random selection. These settings configure ReqLLM's default Finch pool; an explicit `finch: [pools: ...]` configuration takes precedence.

If you need origin-specific pools, HTTP/2, connection options, or pool metrics, configure Finch directly:

Advanced Finch configuration

config :req_llm, finch: [ name: ReqLLM.Finch, pools: %{ :default => [protocols: [:http1], size: 1, count: 32] } ]

Use `pool_timeout: ...` on an individual `stream_text/3` or `stream_object/4` call when one workload needs a longer connection checkout window than the global `stream_pool_timeout` setting.

Advanced users can specify custom Finch instances per request:

API Key Management

ReqLLM makes key management as easy and flexible as possible - this needs to just work.

Please submit a PR if your key management use case is not covered

Keys are pulled from multiple sources with clear precedence: per-request override → in-memory storage → application config → environment variables → .env files.

```elixir

Advanced Req Plugin API

For advanced non-streaming use cases, you can use ReqLLM providers directly as Req plugins. This is the canonical implementation used by ReqLLM.generate_text/3:

```elixir

Customize the Req pipeline with additional headers or middleware

{:ok, model} = ReqLLM.model("anthropic:claude-haiku-4-5") {:ok, provider_module} = ReqLLM.provider(model.provider) {:ok, request} = provider_module.prepare_request(:chat, model, "Hello!", temperature: 0.7)