开源AI工作流

CI Setup

The $1,800/mo Example

An AI coding agent builds a RAG pipeline and picks Opus for every step — embeddings, retrieval, summarization, response generation. Those calls will run in production, billed per token to your API key.

Without Inferwise: The agent picks the most capable (and expensive) model for every call. The bill shows up after the code ships. $2,400/mo.

With Inferwise MCP: The agent calls suggest_model for each task — it learns that classification only needs gpt-4o-mini, summarization works fine on claude-sonnet-4, and only the reasoning step needs claude-opus-4. Cost drops to $600/mo before a single line ships.

With Inferwise CI: Even if the agent doesn't use MCP, inferwise diff flags "+$2,400/mo in new API costs" on the PR. The developer swaps models. Same result.

$1,800/mo saved before a single line ships.

---

Quick Start

```bash

Concrete Example

Your code:
  anthropic.messages.create({
    model: "claude-opus-4-20250514",
    system: "Classify tickets into: billing, technical, account",
    messages: [{ role: "user", content: ticket }],
  })

Inferwise analyzes:
  System prompt extracted -> medium confidence
  No code/reasoning keywords -> capability: [general]
  Opus 4 = premium tier, quality: 94/100, $75/M output

Candidates passing quality gate (general, quality >= 66):
  o3           mid tier, quality: 94, $8/M   -> quality-adj: $8.51/M
  gpt-4.1      mid tier, quality: 90, $8/M   -> quality-adj: $8.89/M
  gpt-4o       mid tier, quality: 77, $10/M  -> quality-adj: $12.99/M

Blocked by quality gate:
  flash-lite   budget, quality: 52 -> 52/94 = 55% < 70% threshold

Result:
  chat-service.ts:8  claude-opus-4 -> o3 (openai)
    Use case: [general] (medium confidence)
    Savings: $527/mo -- quality: 94 vs 94

Every model in the pricing database is tagged with its capabilities and quality benchmarks. See packages/pricing-db/providers/ for pricing and packages/pricing-db/benchmarks.json for quality scores.

---

Set up guardrails: config + git hooks + CI

npx inferwise init

Configuration

`inferwise.config.json`

Create with inferwise init or manually:

{
  "defaultVolume": 1000,
  "ignore": ["node_modules", "dist", "build", "test", "__tests__", "*.test.ts", "*.spec.ts"],
  "overrides": [
    {
      "pattern": "src/chat/**",
      "volume": 5000
    }
  ],
  "budgets": {
    "warn": 2000,
    "block": 50000,
    "requireApproval": 10000,
    "approvers": ["platform-eng", "@infra-team"]
  },
  "telemetry": {
    "backend": "grafana-tempo",
    "endpoint": "https://tempo.internal:3200",
    "apiKey": "glsa_..."
  }
}

Budget thresholds (monthly cost increase in USD):

Budget defaults are deliberately high — block is an emergency brake for catastrophic changes (wrong model at scale, missing max_tokens cap), not routine cost increases.

Environment Variables

---

Direct provider APIs

ANTHROPIC_ADMIN_API_KEY=sk-ant-admin-... inferwise calibrate .

Or calibrate ALL providers via OpenRouter (one API key covers everything)

OPENROUTER_API_KEY=sk-or-... inferwise calibrate .

Fetches real usage data from provider APIs (or OpenRouter), computes correction ratios, and stores them locally. Future estimates go from "2-5x accuracy" to "within 20%".

**6. Connect to OTel for production-accurate estimates (optional)**

Add to `inferwise.config.json`:

If your LLM calls are instrumented with OpenTelemetry (using the GenAI semantic conventions), Inferwise reads your existing traces to get real token averages per model. Estimates go from "2-5x accuracy" to "within 10%". Supports Grafana Tempo and Prometheus/OTLP backends.

---

What Ships: Four Packages

---

End-to-End Pipeline

Every AI API call in your codebase can pass through four tiers before it reaches production.

Code written (by human or AI agent)
        |
        v
  +-----------+
  |  TIER 0   |  Smart model selection (before/during code writing)
  |           |  MCP suggest_model / inferwise audit
  |           |  "Use gpt-4o-mini — classification doesn't need Opus"
  +-----+-----+
        |
        v
  +-----------+
  |  TIER 1   |  Pre-commit hook (developer machine)
  |           |  inferwise estimate .
  |           |  "This commit adds $2,400/mo in LLM costs"
  +-----+-----+
        |  developer pushes
        v
  +-----------+
  |  TIER 2   |  CI gate (GitHub Action / GitLab / any CI)
  |           |  inferwise diff --base main --head HEAD
  |           |  Posts cost report on PR, applies labels
  +-----+-----+
        |  budget check
        v
  +-----------+
  |  TIER 3   |  Budget policy (inferwise.config.json)
  |           |  warn: $2,000   -> yellow label, warning in PR
  |           |  block: $50,000 -> exit code 1, fails CI, blocks merge
  +-----------+

For Developers: The Day-to-Day Workflow

1. Setup (once)

npx inferwise init

Creates inferwise.config.json, installs a pre-commit hook, prints CI setup instructions for GitHub Actions / GitLab / Bitbucket / Jenkins.

2. Audit existing code for optimization

npx inferwise audit .

SMART MODEL ALTERNATIVES

  src/rag.ts:6 — claude-opus-4 → claude-sonnet-4 (anthropic)
    Use case: general (high confidence)
    Reason: Task requires [general] — Sonnet handles that at 79% savings
    Savings: $14,595/mo ($18,440 → $3,845)

  src/classify.ts:14 — gpt-4o → gpt-4o-mini (openai)
    Use case: general (medium confidence)
    Reason: Task requires [general] — gpt-4o-mini handles that at 90% savings
    Savings: $4,500/mo ($5,000 → $500)

Inferwise reads the prompts in your code, infers what each LLM call does, and recommends cheaper models that can handle the task. See How Model Selection Works for details.

3. Write code with LLM API calls

You (or an AI coding agent) write code that calls provider APIs. On git commit, the pre-commit hook runs automatically:

$ git commit -m "feat: add summarizer"

File               Line  Provider   Model           Cost/Call  Monthly
src/summarize.ts   18    openai     gpt-4o          $0.0064    $192/mo
src/rag.ts         91    anthropic  claude-opus-4   $0.0429    $1,287/mo

Total: $1,479/mo (at 1,000 req/day)

You see the cost impact before the code leaves your machine.

4. Open a pull request

CI runs inferwise diff. The GitHub Action posts a cost report directly on the PR:

Net: +$1,242/mo

If the increase exceeds budgets.block, the PR is blocked from merging.

5. Calibrate for tighter estimates (optional)

```bash

Bitbucket Pipelines

pipelines:
  pull-requests:
    '**':
      - step:
          name: Cost Check
          script:
            - npx inferwise diff --format table

Compare costs between branches

npx inferwise diff

For AI agents — add the MCP server:

Cursor / VS Code / Windsurf — add to MCP settings:

{ "mcpServers": { "inferwise": { "command": "npx", "args": ["-y", "@inferwise/mcp"] } } }

Or install globally:

Cursor / VS Code / Windsurf — add to MCP settings:

{ "mcpServers": { "inferwise": { "command": "npx", "args": ["-y", "@inferwise/mcp"] } } }

Once connected, the agent gets four tools:

| Tool | What It Does |
|------|-------------|
| `suggest_model` | Describe a task, get back the cheapest capable model with alternatives and reasoning |
| `estimate_cost` | Estimate the cost of an LLM API call given provider, model, and token counts |
| `audit` | Scan a directory for LLM API calls and suggest cheaper capable alternatives |
| `apply_recommendations` | Auto-apply model swap recommendations to source files (from audit or explicit list) |

**Example flow — new code:** An agent writing a classification pipeline calls `suggest_model` with task "classify support tickets by category" — Inferwise returns `gpt-4o-mini` at $0.60/MTok instead of `gpt-4o` at $10/MTok. The agent writes the code with the right model from the start.

**Example flow — existing code:** An agent calls `apply_recommendations` with `{ directory: "." }` — Inferwise audits the codebase, finds that `claude-opus-4` is overkill for a classification task, and rewrites the model ID to `claude-sonnet-4` in the source file. The agent commits the change. No human needed.

The MCP server runs locally as a subprocess — no hosted infrastructure, no API keys needed. It communicates via stdio using JSON-RPC. Works with Claude Code, Cursor, VS Code (1.99+), Windsurf, Cline, and any MCP-compatible tool.

#### Programmatic Alternatives

For agents and pipelines that don't support MCP:

**SDK** — embed directly in agent pipelines:

const result = await estimateAndCheck("./src", { maxMonthlyCost: 10000, volume: 5000 }); if (!result.ok) console.error("Over budget:", result.violations);

**CLI** — tool-use for agents and scripts:

**Pricing database** — for model routers and cost-aware selection:

const suggestion = suggestModelForTask("classify support tickets"); const cost = calculateCost({ model: suggestion.model, inputTokens: 2000, outputTokens: 500 }); ```

---