AgentHub

Installation

Basic Usage

[!NOTE] We recommend using the stateful interface when calling the MMSP SDK.

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': 'Hello'}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ','}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ' World'}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': '!'}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.done', 'text': 'Hello, World!'}], 'usage_metadata': None, 'finish_reason': None}

{'role': 'assistant', 'event_type': 'stop', 'content_items': [], 'usage_metadata': {'cached_tokens': 0, 'prompt_tokens': 12, 'thoughts_tokens': 0, 'response_tokens': 8}, 'finish_reason': 'stop'}

TypeScript Example:

process.env.OPENAI_API_KEY = "your-openai-api-key";

async function main() { const client = new AutoLLMClient({ model: "gpt-5.6-sol" }); for await (const event of client.streamingResponseStateful({ message: { role: "user", content_items: [{ type: "text.done", text: "Say 'Hello, World!'" }] }, config: {} })) { console.log(event); } }

main().catch(console.error); // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': 'Hello'}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ','}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ' World'}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': '!'}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.done', 'text': 'Hello, World!'}], 'usage_metadata': null, 'finish_reason': null} // {'role': 'assistant', 'event_type': 'stop', 'content_items': [], 'usage_metadata': {'cached_tokens': 0, 'prompt_tokens': 12, 'thoughts_tokens': 0, 'response_tokens': 8}, 'finish_reason': 'stop'} ```

Token Usage

MMSP provides detailed token usage information through the usage_metadata field of the stop event, the last event of every stream.

The usage_metadata object contains four fields: - cached_tokens: Cached input tokens - prompt_tokens: Non-cached input tokens - thoughts_tokens: Chain-of-thought output tokens - response_tokens: Non-chain-of-thought output tokens

You can calculate the total token usage as follows: - input_tokens = cached_tokens + prompt_tokens - output_tokens = thoughts_tokens + response_tokens - total_tokens = input_tokens + output_tokens

█████████████  ░░░░░░░░░░░░░ → LLM → ███████████████  ░░░░░░░░░░░░░░░
cached_tokens  prompt_tokens         thoughts_tokens  response_tokens
        input_tokens                          output_tokens

Concepts: UniConfig, UniMessage and UniEvent

UniConfig

UniConfig is an object that contains the configuration for LLMs.

Example UniConfig:

{
  "max_tokens": 1024,
  "temperature": 1.0,
  "tools": [
    {
      "name": "get_current_weather",
      "description": "Get the current weather in a given location",
      "parameters": {
          "type": "object",
          "properties": {
              "location": {
                  "type": "string",
                  "description": "The city and state, e.g. San Francisco, CA"
              }
          },
          "required": ["location"]
      }
    }
  ],
  "thinking_summary": true,
  "thinking_level": "none | low | medium | high | xhigh | max",
  "tool_choice": "auto | required | none | a list of allowed tool names",
  "system_prompt": "You are a helpful assistant.",
  "prompt_caching": "enable | disable | enhance",
  "fast_mode": false,
  "image_config": {"aspect_ratio": "4:3", "image_size": "1K"},
  "tts_config": [{"voice": "Kore"}],
  "embedding_config": {"dimensions": 768},
  "trace_id": null
}

APIs

AutoLLMClient is the main class for interacting with the MMSP SDK. It is constructed with model and client_type (see Clients; a model id of a known family names its official client on its own), plus optional api_key, base_url, and default_headers — headers sent with every request, for endpoints that demand their own. It provides the following methods:

A key goes only where it was given for: a client reads OPENAI_API_KEY / ANTHROPIC_API_KEY from the environment only together with OPENAI_BASE_URL / ANTHROPIC_BASE_URL (or the provider's own endpoint), so a base_url passed in needs an api_key passed in with it, or the client raises at construction. A vendor client (deepseek-v4, glm-5.x, kimi-k*, minimax-m3, Gemini) reads its own variable, whatever endpoint it is given.

• (async) streaming_response(messages, config): Streams the response of LLMs in a stateless manner.
• (async) streaming_response_stateful(message, config): Streams the response of LLMs in a stateful manner.
• (async) list_models(): Lists the model ids the configured endpoint serves. A protocol client (openai-chat, openai-chat-vllm-adapter, openai-responses, ant-messages, openai-embedding, google-genai, mmsp) is named explicitly and lists everything the endpoint serves; a client deduced from a model id lists only the ids that deduce back to it.
• clear_history(): Clears the history of the stateful LLM client.
• get_history(): Returns the history of the stateful LLM client.
• set_history(history): Replaces the history of the stateful LLM client with a copy of the provided list.

Both streaming methods yield delta events, each carrying one content item, followed by exactly one stop event that carries the usage and the finish reason (see UniEvent).

Streaming clients skip output they do not recognize, so a gateway's own frames cannot end a generation. Set MMSP_DEBUG to anything other than 0, false, no or off to make it raise instead.

SiliconFlow Qwen3.8 27B via OpenAI-compatible API

<details><summary><strong>Python Example</strong></summary>

import asyncio
import os
from mmsp import AutoLLMClient

os.environ["OPENAI_API_KEY"] = "your-siliconflow-api-key"
os.environ["OPENAI_BASE_URL"] = "https://api.siliconflow.cn/v1"

async def main():
    client = AutoLLMClient(model="Qwen/Qwen3.8-27B", client_type="openai-chat")
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
        },
        config={}
    ):
        print(event)

asyncio.run(main())

</details> <details><summary><strong>TypeScript Example</strong></summary>

import { AutoLLMClient } from "@prismshadow/mmsp";

process.env.OPENAI_API_KEY = "your-siliconflow-api-key";
process.env.OPENAI_BASE_URL = "https://api.siliconflow.cn/v1";

async function main() {
  const client = new AutoLLMClient({
    model: "Qwen/Qwen3.8-27B",
    clientType: "openai-chat",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text.done", text: "Say 'Hello, World!'" }],
    },
    config: {}
  })) {
    console.log(event);
  }
}

main().catch(console.error);

SiliconFlow Qwen3 Embedding 0.6B via OpenAI-compatible API

<details><summary><strong>Python Example</strong></summary>

import asyncio
import os
from mmsp import AutoLLMClient

os.environ["OPENAI_API_KEY"] = "your-siliconflow-api-key"
os.environ["OPENAI_BASE_URL"] = "https://api.siliconflow.cn/v1"

async def main():
    client = AutoLLMClient(model="Qwen/Qwen3-Embedding-0.6B", client_type="openai-embedding")

    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Hello world"}],
        },
        config={},
    ):
        print(event)

asyncio.run(main())

<details><summary><strong>TypeScript Example</strong></summary>

import { AutoLLMClient } from "@prismshadow/mmsp";

process.env.OPENAI_API_KEY = "your-siliconflow-api-key";
process.env.OPENAI_BASE_URL = "https://api.siliconflow.cn/v1";

async function main() {
  const client = new AutoLLMClient({
    model: "Qwen/Qwen3-Embedding-0.6B",
    clientType: "openai-embedding",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text.done", text: "Hello world" }],
    },
    config: {},
  })) {
    console.log(event);
  }
}

main().catch(console.error);

Supported Models

Python package

Install from PyPI:

```bash uv add mmsp

TypeScript package

Install from npm:

npm install @prismshadow/mmsp

Build from source:

cd src_ts && make install && make build

See src_ts/README.md for comprehensive usage examples and API documentation.