Foveance

Run with Docker (Recommended)

You can run the Foveance proxy with zero Python setup using Docker:

docker run -p 8799:8799 ghcr.io/aimaghsoodi/foveance --upstream https://api.openai.com/v1

Library usage (beyond `shrink`)

from foveance import Controller, Item
from foveance.llm import MockLLM   # or OllamaLLM("gemma2:9b"), OpenAICompatLLM(...)

ctrl = Controller(MockLLM(), budget=2000, policy="foveance", drift=0.7)
ctrl.add_item(Item("obs0", "tool_output", "FACT api_key=sk-123\n...lots of logs...", created_turn=0))
rec = ctrl.step("recall api_key", turn=0)
print(rec.answer, rec.input_tokens, rec.peak_tokens)

The public API at a glance (from foveance import ...):

Option A — you use a coding agent (Claude Code, Codex, aider, …)

One command. It runs your tool exactly as before, just cheaper, and prints how much you saved:

pip install foveance
foveance wrap claude          # or:  foveance wrap -- codex "fix the tests"

<p align="center"><img alt="foveance wrap demo — 3,590 to 1,677 input tokens, -53%, same answer" src="https://raw.githubusercontent.com/aimaghsoodi/foveance/main/assets/demo.gif" width="640"></p>

That's the whole thing. Your API key is untouched, nothing is stored, and a live "tokens saved ≈ $" dashboard runs at <http://localhost:8799/> while you work.

Option B — you write Python

One install, one function. No server, no config, nothing to run:

pip install foveance

smaller = shrink(messages, budget=2000) # messages = your OpenAI-style list

Option C — try it right now, no API key, no GPU

pip install foveance
foveance demo

---

then point any client at it with one variable (your API key still goes straight upstream):

export OPENAI_BASE_URL=http://localhost:8799/v1 # OpenAI SDK, Codex, Ollama-backed apps export ANTHROPIC_BASE_URL=http://localhost:8799 # Anthropic SDK, Claude Code ```

Works with anything that lets you set its base URL — the OpenAI and Anthropic SDKs, Claude Code, Codex (with an API key), aider, Continue, Cursor, LangChain, LiteLLM, and local runtimes like Ollama / vLLM / LM Studio. Foveance is auth-free: it adds no login of its own and stores no key. The only thing it can't intercept is a client that cryptographically hard-pins its endpoint (e.g. ChatGPT-subscription Codex); give such a tool an API key and it works like everything else.

It listens on http://localhost:8799 and exposes POST /v1/chat/completions, POST /v1/messages, POST /v1/responses, GET /v1/models, GET /health, GET /admin/stats (JSON), and a live dashboard at GET / (tokens saved and ≈$ at --price-per-mtok). "stream": true is passed through verbatim. Plain chat is compressed by the anticipatory allocator; tool-using (agentic) requests are compressed structurally in place, preserving every message and tool-call pairing.

Prompt-cache aware: blocks carrying an Anthropic cache_control breakpoint are never modified, and with --cache-aware the proxy never touches anything at or before the last breakpoint — so it never invalidates the provider's prompt cache. See docs/limitations.md for the cost arithmetic.

<p align="center"><img alt="foveance works with Claude Code, Codex, Ollama, and any OpenAI/Anthropic-compatible tool" src="https://raw.githubusercontent.com/aimaghsoodi/foveance/main/assets/demo_anytool.gif" width="640"></p>

Head-to-head vs other methods (real model + real LLMLingua-2)

A long trajectory hides one load-bearing fact early amid filler; each method compresses to a budget, then the real model (llama3.2:1b) is asked to recall it. Only the query-aware allocators recall it at every budget, at 5–10× fewer tokens than full replay:

The same ordering holds in the full multi-turn agent loop, ruling out a one-shot artifact:

Reproduce: python bench/compare_baselines.py --with-llmlingua && python bench/plot_baselines.py. LLMLingua-2 is a real run via the llmlingua package (CPU).