# LLM disk cache — status & operator guide (2026-05-14)

Wave 2.1 of the framework-hardening roadmap. Disk-cached LLM
responses for `AsyncLLMClient.complete()`.

## What it does

Every `await client.complete(...)` call that returns successfully
writes its response into `data/llm_cache.db` keyed by SHA256 of
`(provider, model, messages, temperature, max_tokens, response-shaping kwargs)`.
A subsequent call with identical inputs returns from the SQLite file in
<10 ms instead of hitting the provider.

## When it pays off

- **Re-running tests / debugging prompts**: same prompt, same answer,
  no API charge.
- **Agent-CLI paths**: a 6–16 s Codex / Claude Code round-trip
  collapses to a single SQLite read.
- **Multi-stage pipelines**: when a single user query triggers
  routing + screening + rephrase + retrieval + synthesis, the first
  three are typically deterministic and rarely change between runs —
  perfect cache fits.

## When to bypass

```python
await client.complete(..., cache=False)
```

- Liveness / integration tests that need genuine round-trips.
- High-temperature draws where you actually want a fresh sample.
- Debugging "is this provider drift or my code?" — bypass once, see
  whether the answer changes.

## Configuration

```yaml
llm:
  cache_enabled: true               # default true; false disables read+write
  cache_path: data/llm_cache.db     # default; SQLite file
  cache_ttl_hours: 24               # default 24h; 0 = never expire
```

## Clearing the cache

There is no admin CLI yet (followup). To clear manually:

```bash
rm data/llm_cache.db data/llm_cache.db-shm data/llm_cache.db-wal
```

Or selectively, via SQLite:

```bash
sqlite3 data/llm_cache.db "DELETE FROM llm_cache WHERE provider = 'anthropic';"
```

## What is NOT cached

- `stream()` calls — chunked output is rare in this codebase and the
  deterministic-replay value is low.
- Embedding calls — covered by Wave 2.2 separately.
- Anthropic server-side ephemeral cache_control — that's a separate
  optimisation layer; the two compose (disk cache hits *before* HTTP).

## Provenance integration

Cache hits still appear in the provenance collector with the original
`latency_ms` and `input_tokens` / `output_tokens`. They are
indistinguishable from a fresh call in the trace — except they
happened in <10 ms. (A future Wave 2.4 budget-caps pass may add an
explicit `cached: True` flag so cost accounting can skip them.)

## Files

| File | Purpose |
|---|---|
| `src/perspicacite/llm/cache.py` | `LLMResponseCache` + `build_cache_key` |
| `src/perspicacite/llm/client.py` | Wiring into `AsyncLLMClient.complete` |
| `src/perspicacite/config/schema.py` | `cache_enabled`, `cache_path`, `cache_ttl_hours` |
| `tests/unit/test_llm_cache_key.py` | Key-stability tests |
| `tests/unit/test_llm_cache_storage.py` | SQLite layer tests |
| `tests/unit/test_llm_client_cache_integration.py` | End-to-end wiring tests |

## Open followups

- Per-provider TTL overrides (only if drift patterns warrant).
- `cache_admin` CLI (grouped with Wave 2.4 budget caps).
- LRU size cap (only if disk grows past 1 GB).
- Cache `stream()` for completeness (low priority).
