---
title: Tracked LLM Calls
description: Use kitaru.llm() with model aliases, transported runtime config, and optional secret-backed credentials
---

`kitaru.llm()` lets you make a single tracked model call with automatic:

- prompt artifact capture
- response artifact capture
- usage/latency metadata logging

<Callout type="info">
  If you want the full setup path from stored credentials to an actual flow run,
  start with
  [Secrets + Model Registration](/guides/secrets-and-model-registration).
</Callout>

## Model selection order

When you call `kitaru.llm()`, Kitaru resolves the model in this order:

1. the explicit `model=` argument
2. `KITARU_DEFAULT_MODEL`
3. the default alias from the effective model registry in the current environment

If `KITARU_DEFAULT_MODEL` matches a registered alias, Kitaru resolves that
alias. Otherwise it treats the value as a raw provider/model string.

When you submit or replay a flow, Kitaru automatically transports your local
model registry into the execution environment. That means remote runs can still
resolve aliases with `kitaru.llm()` and `kitaru model list`. If
`KITARU_MODEL_REGISTRY` is already set in the runtime environment, its aliases
and default alias take precedence over matching local entries.

## Register a model alias

```bash
kitaru model register fast --model openai/gpt-5-nano --secret openai-creds
```

You can also register an alias without a linked secret:

```bash
kitaru model register fast --model openai/gpt-5-nano
```

List aliases with:

```bash
kitaru model list
```

<Callout type="info">
  `kitaru model register` writes aliases to local Kitaru config, but submitted
  and replayed runs automatically receive that registry as a transported runtime
  snapshot. `KITARU_MODEL_REGISTRY` is available as an advanced manual override
  for adding aliases or overriding matching ones.
</Callout>

## Supported providers

Built-in runtime support covers:

- `openai/*` — OpenAI models (requires `kitaru[openai]`)
- `anthropic/*` — Anthropic models (requires `kitaru[anthropic]`)
- `ollama/*` — local Ollama models (requires `kitaru[openai]`, no API key needed)
- `openrouter/*` — OpenRouter meta-router (requires `kitaru[openai]`)

Ollama and OpenRouter use the OpenAI-compatible API, so they share the
`kitaru[openai]` extra — no additional packages needed.

## Credential resolution order

For built-in providers that require credentials (OpenAI, Anthropic,
OpenRouter), Kitaru resolves credentials in this order:

1. provider credentials already present in the environment
2. the secret linked to the resolved alias
3. otherwise, fail with a setup error

That means environment variables win over a linked secret for known providers.

Ollama does not require credentials (local server). Use `OLLAMA_HOST` to
point to a non-default server address (default: `http://localhost:11434`).

### Environment-backed setup

```bash
export OPENAI_API_KEY=sk-...
```

### Secret-backed setup

Store provider keys in a Kitaru secret:

```bash
kitaru secrets set openai-creds --OPENAI_API_KEY=sk-...
```

When an alias includes `--secret openai-creds`, `kitaru.llm()` loads that
secret at runtime if the required environment variable is not already set.

## Call `kitaru.llm()` inside a flow

```python
from kitaru import flow
import kitaru

@flow
def writer(topic: str) -> str:
    outline = kitaru.llm(
        f"Create a 3-bullet outline about {topic}.",
        model="fast",
        name="outline_call",
    )
    outline_text = outline.load()
    return kitaru.llm(
        f"Write a short paragraph using this outline:\n{outline_text}",
        model="fast",
        name="draft_call",
    )
```

Flow-body `kitaru.llm()` calls are durable call boundaries. Use `.load()` when
you need the text in flow-body Python, such as composing the next prompt. If you
pass a checkpoint or LLM output into a downstream checkpoint, keep passing the
original output handle. See [In flow bodies](/guides/artifacts#in-flow-bodies)
for the general pattern.

## Advanced options

`kitaru.llm()` also accepts `system=`, `temperature=`, and `max_tokens=`:

```python
reply = kitaru.llm(
    "Summarize this document in 3 bullets.",
    model="fast",
    system="You are a concise technical editor.",
    temperature=0.2,
    max_tokens=200,
    name="summary_call",
)
```

### Chat-style message lists

Instead of a plain string, you can pass a chat-style message list:

```python
reply = kitaru.llm(
    [
        {"role": "user", "content": "Draft a release note headline."},
        {"role": "assistant", "content": "Kitaru adds durable replay controls."},
        {"role": "user", "content": "Now make it shorter."},
    ],
    model="fast",
    name="headline_refine",
)
```

Each message must include `role` and `content` keys. If `system=` is provided
alongside a message list, Kitaru prepends a system message automatically.

## When to use `kitaru.llm()` vs your own client

`kitaru.llm()` is designed for simple text-in/text-out model calls. It handles
credential resolution, prompt/response capture, and usage tracking automatically.
Built-in runtime support covers `openai/*`, `anthropic/*`, `ollama/*`, and
`openrouter/*` models.

<Callout type="info">
  `kitaru.llm()` requires a provider SDK to be installed. Install with
  `pip install kitaru[openai]` (also covers Ollama and OpenRouter),
  `pip install kitaru[anthropic]`, or `pip install kitaru[llm]` for both.
</Callout>

For advanced patterns — tool calling, structured outputs, streaming, vision
inputs, or multi-turn conversation management — use your provider SDK directly
inside a `@checkpoint`. You still get durable checkpointing and replay; you
just manage the model interaction yourself:

```python
from openai import OpenAI
from kitaru import checkpoint

@checkpoint
def agent_step(messages: list[dict]) -> str:
    client = OpenAI()
    resp = client.chat.completions.create(
        model="gpt-5-nano",
        messages=messages,
        tools=[...],  # tool calling, structured output, etc.
    )
    return resp.choices[0].message.content
```

For a full example of a tool-calling agent built this way, see
[`examples/end_to_end/coding_agent/`](https://github.com/zenml-io/kitaru/tree/develop/examples/end_to_end/coding_agent).

<Callout type="info">
  Tool calling and structured output support for `kitaru.llm()` is on the
  roadmap. For now, use your provider SDK directly inside checkpoints for
  these patterns.
</Callout>

## Runtime behavior by context

- **Inside a flow (outside checkpoints):** `kitaru.llm()` runs as a synthetic durable call boundary.
- **Inside a checkpoint:** it is tracked as a child event; the enclosing checkpoint remains the replay boundary.

## What Kitaru records

Each `kitaru.llm()` call records:

- prompt artifacts
- response artifacts
- token usage
- latency
- credential source metadata (`environment` or `secret`)

## Example in this repository

```bash
uv sync --extra local --extra llm

# Register an alias (with or without a linked secret) before running the example.
uv run kitaru model register fast --model openai/gpt-5-nano
uv run python examples/features/llm/flow_with_llm.py
uv run pytest tests/test_phase12_llm_example.py
```

If you want the full credential-backed setup path first, start with
[Secrets + Model Registration](/guides/secrets-and-model-registration).

For the broader catalog, see [Examples](/getting-started/examples).

## Related pages

- [Examples](/getting-started/examples)
- [Secrets + Model Registration](/guides/secrets-and-model-registration)
- [Manage Secrets](/guides/secrets)
- [Configuration](/guides/configuration)
- [CLI model commands](/cli/model)
