AgentHub

Installation

Basic Usage

[!NOTE] We recommend using the stateful interface when calling the AgentHub SDK.

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': 'Hello'}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': ','}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': ' World'}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': '!'}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'stop', 'content_items': [], 'usage_metadata': {'cached_tokens': 0, 'prompt_tokens': 12, 'thoughts_tokens': 0, 'response_tokens': 8}, 'finish_reason': 'stop'}

TypeScript Example:

process.env.OPENAI_API_KEY = "your-openai-api-key";

async function main() { const client = new AutoLLMClient({ model: "gpt-5.6-sol" }); for await (const event of client.streamingResponseStateful({ message: { role: "user", content_items: [{ type: "text", text: "Say 'Hello, World!'" }] }, config: {} })) { console.log(event); } }

main().catch(console.error); // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': 'Hello'}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': ','}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': ' World'}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text', 'text': '!'}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'stop', 'content_items': [], 'usage_metadata': {'cached_tokens': 0, 'prompt_tokens': 12, 'thoughts_tokens': 0, 'response_tokens': 8}, 'finish_reason': 'stop'} ```

Token Usage

AgentHub provides detailed token usage information through the usage_metadata field in streaming events.

The usage_metadata object contains four fields: - cached_tokens: Cached input tokens - prompt_tokens: Non-cached input tokens - thoughts_tokens: Chain-of-thought output tokens - response_tokens: Non-chain-of-thought output tokens

You can calculate the total token usage as follows: - input_tokens = cached_tokens + prompt_tokens - output_tokens = thoughts_tokens + response_tokens - total_tokens = input_tokens + output_tokens

█████████████  ░░░░░░░░░░░░░ → LLM → ███████████████  ░░░░░░░░░░░░░░░
cached_tokens  prompt_tokens         thoughts_tokens  response_tokens
        input_tokens                          output_tokens

Concepts: UniConfig, UniMessage and UniEvent

UniConfig

UniConfig is an object that contains the configuration for LLMs.

Example UniConfig:

{
  "max_tokens": 1024,
  "temperature": 1.0,
  "tools": [
    {
      "name": "get_current_weather",
      "description": "Get the current weather in a given location",
      "parameters": {
          "type": "object",
          "properties": {
              "location": {
                  "type": "string",
                  "description": "The city and state, e.g. San Francisco, CA"
              }
          },
          "required": ["location"]
      }
    }
  ],
  "thinking_summary": true,
  "thinking_level": "none | low | medium | high | xhigh | max",
  "tool_choice": "auto | required | none | a list of allowed tool names",
  "system_prompt": "You are a helpful assistant.",
  "prompt_caching": "enable | disable | enhance",
  "fast_mode": false,
  "image_config": {"aspect_ratio": "4:3", "image_size": "1K"},
  "tts_config": [{"voice": "Kore"}],
  "embedding_config": {"dimensions": 768},
  "trace_id": null
}

AgentHub SDK - Unified and Precise LLM SDK

AgentHub is the LLM API Hub for the Agent era, built for high-precision autonomous agents.

Using a coding agent? Install the AgentHub SKILL files from skills/ so it can use AgentHub correctly in generated code.

📢 Follow us on X: or join our Discord Community

APIs

AutoLLMClient is the main class for interacting with the AgentHub SDK. It is constructed with model, plus optional api_key, base_url, client_type, and default_headers — headers sent with every request, for endpoints that demand their own. It provides the following methods:

• (async) streaming_response(messages, config): Streams the response of LLMs in a stateless manner.
• (async) streaming_response_stateful(message, config): Streams the response of LLMs in a stateful manner.
• (async) list_models(): Lists the model ids the configured endpoint serves. A protocol client (openai-chat, openai-chat-vllm-adapter, openai-responses, ant-messages, openai-embedding) is named explicitly and lists everything the endpoint serves; a client deduced from a model id lists only the ids that deduce back to it.
• clear_history(): Clears the history of the stateful LLM client.
• get_history(): Returns the history of the stateful LLM client.
• set_history(history): Replaces the history of the stateful LLM client with a copy of the provided list.

Streaming clients skip output they do not recognize, so a gateway's own frames cannot end a generation, and an event a client has nothing universal to emit for never reaches you. Set AGENTHUB_DEBUG to anything other than 0, false, no or off to make both raise instead.

SiliconFlow Qwen3.6 35B via OpenAI-compatible API

<details><summary><strong>Python Example</strong></summary>

import asyncio
import os
from agenthub import AutoLLMClient

os.environ["OPENAI_API_KEY"] = "your-siliconflow-api-key"
os.environ["OPENAI_BASE_URL"] = "https://api.siliconflow.cn/v1"

async def main():
    client = AutoLLMClient(model="Qwen/Qwen3.6-35B-A3B", client_type="openai-chat")
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text", "text": "Say 'Hello, World!'"}]
        },
        config={}
    ):
        print(event)

asyncio.run(main())

</details> <details><summary><strong>TypeScript Example</strong></summary>

import { AutoLLMClient } from "@prismshadow/agenthub";

process.env.OPENAI_API_KEY = "your-siliconflow-api-key";
process.env.OPENAI_BASE_URL = "https://api.siliconflow.cn/v1";

async function main() {
  const client = new AutoLLMClient({
    model: "Qwen/Qwen3.6-35B-A3B",
    clientType: "openai-chat",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text", text: "Say 'Hello, World!'" }],
    },
    config: {}
  })) {
    console.log(event);
  }
}

main().catch(console.error);

SiliconFlow Qwen3 Embedding 0.6B via OpenAI-compatible API

<details><summary><strong>Python Example</strong></summary>

import asyncio
import os
from agenthub import AutoLLMClient

os.environ["OPENAI_API_KEY"] = "your-siliconflow-api-key"
os.environ["OPENAI_BASE_URL"] = "https://api.siliconflow.cn/v1"

async def main():
    client = AutoLLMClient(model="Qwen/Qwen3-Embedding-0.6B", client_type="openai-embedding")

    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text", "text": "Hello world"}],
        },
        config={},
    ):
        print(event)

asyncio.run(main())

<details><summary><strong>TypeScript Example</strong></summary>

import { AutoLLMClient } from "@prismshadow/agenthub";

process.env.OPENAI_API_KEY = "your-siliconflow-api-key";
process.env.OPENAI_BASE_URL = "https://api.siliconflow.cn/v1";

async function main() {
  const client = new AutoLLMClient({
    model: "Qwen/Qwen3-Embedding-0.6B",
    clientType: "openai-embedding",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text", text: "Hello world" }],
    },
    config: {},
  })) {
    console.log(event);
  }
}

main().catch(console.error);

Supported Models

Beyond the model-specific clients, four generic protocol clients call any compatible endpoint:

- client_type="openai-chat" — OpenAI Chat Completions. Bare "openai" is an alias. - client_type="openai-chat-vllm-adapter" — Chat Completions as served by vLLM, mapping thinking_level onto the chat_template_kwargs the served model's template reads. - client_type="openai-responses" — OpenAI Responses, served by OpenAI, OpenRouter, DeepSeek, Z.AI, and MiniMax. - client_type="ant-messages" — Anthropic Messages, served by Anthropic, OpenRouter, DeepSeek, Z.AI, and MiniMax.

Where a gateway serves more than one, prefer "openai-responses": OpenRouter serves it for every model it hosts, while SiliconFlow serves Chat Completions only.

The full machine-readable list — model, base URL, client, input/output modalities, context window, and per-million-token list pricing in USD or CNY:

from agenthub import list_supported_models

models = list_supported_models(currency="CNY")  # "USD" by default

import { listSupportedModels } from "@prismshadow/agenthub";

const models = listSupportedModels("CNY"); // "USD" by default

Python package

Install from PyPI:

```bash uv add agenthub-python

TypeScript package

Install from npm:

npm install @prismshadow/agenthub

Build from source:

cd src_ts && make install && make build

See src_ts/README.md for comprehensive usage examples and API documentation.