AI Mock工具

简介

Features

- Record & Replay — Proxy real APIs, save as fixtures, replay deterministically forever - Timing-aware recording and replay — Recorded fixtures capture per-frame arrival timestamps; replay uses recorded timings for approximate timing reproduction based on recorded TTFT and inter-frame cadence (replay chunk count may differ from recording — TTFT and average pace are preserved, not per-token fidelity) with configurable --replay-speed multiplier - Recorded token usage and cost — Recording captures the final usage frame of a streaming completion, so replayed fixtures serve real prompt_tokens / completion_tokens instead of a length estimate; OpenRouter's provider-reported usage.cost and its cost_details / *_tokens_details breakdowns are captured too, so an app that bills from real provider cost can e2e-test its wallet path from a tape - Multi-turn Conversations — Record and replay multi-turn traces with tool rounds; match distinct turns via turnIndex, hasToolResult, toolCallId, toolResultContains (gate on the tool-result payload), sequenceIndex, systemMessage (gate on host-supplied agent context), or custom predicates - Ollama embedding recordings — Current /api/embed captures retain input shape, dimensions and model in their match and stay separate from legacy/OpenAI embedding operations. - 11 providers across 22 API surfaces — OpenAI Chat, OpenAI Responses, OpenAI Realtime (GA; an OpenAI-Beta: realtime=v1 request gets the real sunset rejection), GPT-Live, Claude, Gemini REST, Gemini embedContent, Gemini Live, Gemini Interactions, Azure, Bedrock, Vertex AI, Ollama (chat + embeddings), Cohere (chat + embed), OpenRouter (chat + router failover), BytePlus Ark (OpenAI-compatible chat + images under /api/v3), ElevenLabs TTS and Voice Design — full streaming support Bedrock with the current AWS JavaScript SDK needs requestHandler: new NodeHttpHandler() from @smithy/node-http-handler for aimock's HTTP/1 server. See SDK setup. - Multimedia APIs — image generation (gpt-image-1 default, Imagen; /v1/images/variations replays OpenAI's removal 404), image editing (/v1/images/edits), text-to-speech (OpenAI + ElevenLabs TTS and Voice Design), audio transcription, audio translation (/v1/audio/translations), video generation, OpenRouter video generation (/api/v1/videos with async job lifecycle), Google Veo video generation (:predictLongRunning + /v1beta/operations async lifecycle), Grok Imagine video generation (/v1/videos/generations with async job lifecycle), BytePlus Ark video generation (Seedance, /api/v3/contents/generations/tasks with async task lifecycle), fal.ai (image / video / audio with queue lifecycle) - OpenAI platform APIs — Files (byte-exact uploads, purpose validation), Vector Stores (CRUD, file attach with deterministic poll progression, file batches, deterministic direct search; hosted file_search execution is not supported), fine-tuning jobs (deterministic lifecycle, events, cursor pages) and Batches (/v1/batches create/list/retrieve/cancel with deterministic poll progression and real output files) - MCP / A2A / AG-UI / Vector — Mock every protocol your AI agents use - MCP Scenario Fakes — An mcpFakes block in a fixture file scripts each test's MCP tool answers next to its LLM turns: ordered answers, exact or any-argument matching, scripted tool errors, and a closed-world mode that fails on any undeclared tool. Works with any MCP client over Streamable HTTP - MCP Recording — Record a live MCP server into mcpFakes files (--mcp-record /mcp=<url>, or llm.record.mcp with llm.enableMcpRecording: true in a config) and replay them offline, with the recorded tool list, progress notifications and timing; known secrets are redacted before anything is written - MCP Fake Reports — GET /__aimock/mcp/fakes/report lists each test's served, unused and failed fakes; with useAimock({ fakesReport: "fail" }) a swallowed fake error or an unused fake fails the test, and fakesFor() gives the MCP URL and headers for the current test. Works for agents, workflows and pipelines alike - Chaos Testing — 500 errors, malformed JSON, mid-stream disconnects and 429 rate limits (--chaos-ratelimit, with Retry-After) at any probability, plus a fixed pre-handling delay (--chaos-latency <ms>); read, set and clear the rates at runtime via GET / POST / DELETE /__aimock/chaos, scoped to the caller's X-Test-Id so one test cannot destabilise the suite running beside it - Model Misbehavior — Test invalid tool arguments, schema violations, unknown tools, duplicate IDs, length stops, empty responses, refusals, content filtering, and reasoning-only responses. Supported wires include OpenAI Chat, Responses, Realtime GA, Anthropic, Bedrock InvokeModel and Converse, Gemini generateContent (including the Vertex wrapper), Gemini Live, Gemini Interactions, Cohere, and Ollama chat. Fault and mode support varies by wire. Azure and OpenRouter use the Chat wire. Seeded rates, per-test limits, and journal evidence apply across supported wires. See the reference for the support matrix, SDK behavior, and modeled contracts whose native occurrence remains unverified. - Request journal — GET /__aimock/journal returns every recorded request, filterable by path, method, status, service, testId and requestId with limit / offset paging and an X-Total-Count header carrying the pre-pagination total - HTTP route catalog — GET /__aimock/openapi.json returns OpenAPI 3.1; GET /__aimock/routes returns { count, routes }. For programmatic inspection, import buildOpenApiDocument, CATALOG_ROUTES and ROUTE_DEFINITIONS (with CatalogRoute/RouteDefinition types) from @copilotkit/aimock. The catalog describes HTTP operations; streaming responses include media and framing metadata, not a complete-stream JSON body. - Request IDs — every response carries X-Request-Id, echoing the caller's header or minting one; the journal stores it on each entry and GET /__aimock/journal?requestId=<id> returns that request. A minted id is never forwarded upstream in record/proxy mode - Per-Request Strict Mode — X-AIMock-Strict header overrides the server-level --strict flag per request (true/1 = strict, false/0 = lenient) - Required Header Policy — Opt-in requiredHeaders: "enforce" (programmatic policy, config llm.policy, repeatable --policy on both CLIs, or the X-AIMock-Policy request header) rejects Anthropic HTTP requests (/v1/messages, /v1/messages/count_tokens, Anthropic-flavored /v1/models/{id}, and the /v1/models list with modelCatalog: "native") that omit anthropic-version with a native 400. Off by default, and independent of strict mode - Native model catalogs and token counts — Anthropic GET /v1/models (selected by anthropic-version, or x-api-key without Authorization; the list route needs policy.modelCatalog: "native" and otherwise stays the OpenAI list) and Gemini GET /v1beta/models list and detail routes with provider paging, built from fixture model IDs with synthetic metadata; POST /v1/messages/count_tokens and Gemini :countTokens return a deterministic estimate of about 1 token per 4 characters of messages (plus tool definitions), minimum 1. These routes never proxy or record - Context-Based Fixture Routing — X-AIMock-Context header scopes fixtures per integration; fixtures with match.context only match requests carrying that context, fixtures without it remain shared - Drift Detection — Daily CI validation against real APIs - Streaming Physics — Configurable ttft, tps, and jitter - WebSocket APIs — OpenAI Realtime (GA protocol with models: gpt-realtime, gpt-realtime-2, gpt-realtime-1.5, gpt-realtime-mini; transcription/translation via gpt-4o-transcribe, gpt-4o-mini-transcribe, whisper-1; image input; commentary phase), Responses WS, Gemini Live - GPT-Live — Mock, record, and replay primary WebSocket sessions at /v1/live/sessions in client and managed modes, with fixture-backed audio and backend tool continuation. Saved fixtures replay offline; aimock does not synthesize speech. This voice surface is separate from OpenAI text-model family support; WebRTC/SDP and sideband connections are outside this scope. - Prometheus Metrics — Request counts, latencies, fixture match rates - Docker + Helm — Container image and Helm chart for CI/CD - Vitest & Jest Plugins — Zero-config useAimock() with auto lifecycle and env patching - Response Overrides — Control id, model, usage, finishReason in fixture responses - Ordered Blocks — A blocks array streams text and tool calls in any order (tool-first or interleaved); blocks-only fixtures are first-class, and the recorder captures order from genuinely tool-first/interleaved streams - Namespaced and Custom Tool Calls — Responses API fixtures can return namespaced function calls and custom tool calls (custom_tool_call) over HTTP streaming, non-streaming and WebSocket, and can match on toolNamespace. This lets you script OpenAI Codex CLI's namespaced MCP and sub-agent tools and its freeform apply_patch tool (with the responsesTools: "extended" server option) - Streaming Usage Chunks — stream_options.include_usage support emits a final chunk with token counts, matching OpenAI's streaming usage protocol - Rate Limiting Headers — x-ratelimit-* headers on every response and Retry-After on 429 errors for testing retry/backoff logic - Zero dependencies — Everything from Node.js builtins

Model misbehavior requires aimock 1.45.0 or later. Older versions silently ignore the fixture's misbehavior field and serve its original response. Invalid configurations now fail during fixture loading or programmatic addition. See the model misbehavior reference for supported parameters and compatibility details.