Streaming Physics

Simulate realistic LLM streaming timing with configurable time-to-first-token (TTFT), tokens-per-second (TPS), and random jitter. Perfect for testing loading states, progress indicators, and streaming UX under realistic conditions.

Streaming timing also paces raw passthrough fixtures, and a non-streaming raw reply follows nonStreamingTiming like a typed fixture (see Raw Passthrough: Timing).

StreamingProfile

The streamingProfile option can be set on any fixture to control the timing of streamed chunks.

Programmatic Usage

const mock = new LLMock();
await mock.start();

// Simulate GPT-4 streaming timing
mock.on(
  { userMessage: "hello" },
  { content: "Hello! How can I help you today?" },
  {
    streamingProfile: {
      ttft: 800,    // 800ms before first token
      tps: 50,      // 50 tokens/sec after that
      jitter: 0.2,  // +/-20% variance on each delay
    },
  },
);

JSON Fixture File

{
  "fixtures": [
    {
      "match": { "userMessage": "think carefully" },
      "response": { "content": "Let me think about this..." },
      "streamingProfile": {
        "ttft": 2000,
        "tps": 30,
        "jitter": 0.1
      }
    }
  ]
}

Non-Streaming Responses

By default, streamingProfile and recordedTimings delay only streamed (SSE) replies. A stream: false reply returns immediately, also when the fixture sets either field. This includes fixtures that record mode wrote with recordedTimings.

To also delay stream: false OpenAI Chat Completions replies, set nonStreamingTiming: true on the fixture, next to its streamingProfile or recordedTimings. The delay then applies to text, tool call, combined content and tool call, and blocks replies. aimock waits for the sum of the per-chunk delays that a streamed reply at the same chunkSize would get (ttft, then a tps gap for each further content chunk), then writes the JSON body. A streamed reply also pays for its role and finish chunks, so the two totals are close but not identical.

{
  "fixtures": [
    {
      "match": { "userMessage": "think carefully" },
      "response": { "content": "Let me think about this..." },
      "streamingProfile": { "ttft": 800, "tps": 50 },
      "nonStreamingTiming": true
    }
  ]
}

nonStreamingTiming has an effect only on OpenAI Chat Completions, and only when the fixture also sets streamingProfile or recordedTimings. Non-streaming responses from other providers are not delayed by it.

All Providers

To delay stream: false replies on other providers too, set nonStreamingTimingAllProviders: true on the fixture. It uses the same formula as nonStreamingTiming: true, and it also needs a streamingProfile or recordedTimings. It applies to these non-streaming endpoints:

• OpenAI Chat Completions (the same delay as nonStreamingTiming: true)
• Anthropic Messages
• OpenAI Responses
• Gemini and Vertex AI generateContent
• Gemini Interactions (with stream: false)
• Cohere v2 chat
• Bedrock InvokeModel and Converse
• Ollama /api/chat and /api/generate

Audio transcription replies are not delayed. nonStreamingTiming is unchanged: it still applies to OpenAI Chat Completions only. Setting nonStreamingTimingAllProviders: false does not turn off nonStreamingTiming: true.

{
  "fixtures": [
    {
      "match": { "userMessage": "think carefully" },
      "response": { "content": "Let me think about this..." },
      "streamingProfile": { "ttft": 800, "tps": 50 },
      "nonStreamingTimingAllProviders": true
    }
  ]
}

Errors and Disconnects

• Error replies (a fixture error response) are never delayed. aimock sends them at once, with the fixture status.
• If the client disconnects during the wait, aimock writes nothing. The journal entry records response.interrupted: true, response.interruptReason: "client aborted" and response.status: 0. See Client Disconnects.

Interaction with latency

• When streamingProfile is set, it takes priority over the latency field.
• If streamingProfile is not set, the existing latency behavior applies (flat delay per chunk).
• If streamingProfile is set but has neither ttft nor tps, it falls back to latency.
• latency and --latency are gaps between SSE chunks only. They never delay a non-streaming response, including the fallback case above. To delay stream: false replies, set streamingProfile and nonStreamingTiming: true (OpenAI Chat Completions) or nonStreamingTimingAllProviders: true (see Non-Streaming Responses).

Realistic Profiles

Here are some example profiles that approximate real-world LLM behavior:

// Fast model (GPT-4o-mini, Claude 3 Haiku)
{ ttft: 200, tps: 100, jitter: 0.15 }

// Standard model (GPT-4o, Claude 3.5 Sonnet)
{ ttft: 500, tps: 60, jitter: 0.2 }

// Reasoning model (o1, o3, Claude with extended thinking)
{ ttft: 5000, tps: 80, jitter: 0.1 }

// Slow/overloaded (rate-limited or cold start)
{ ttft: 3000, tps: 15, jitter: 0.4 }