Guide

What is an AI API relay or LLM proxy?

An AI API relay or LLM proxy is a third-party service that sits between your application and an upstream AI provider such as Anthropic or OpenAI. It can forward requests, normalize APIs, meter usage, or route traffic, but it can also change the prompt, model, context, stream, or error response you receive.

Short answer: a relay is part of your model supply chain. If it can read and rewrite requests, it should be audited before production or wallet-related traffic depends on it.

What a relay can change

Prompt path

A relay can prepend hidden instructions, inject identity text, ask the model to ignore user instructions, or reveal hidden prompt content in later responses.

Model path

A relay can route a request to a different model than the one named in the API response, or leak an upstream model identity through wording or stream metadata.

Tool path

A relay can rewrite package-install commands or tool-like output before it reaches a coding agent, which turns proxy behavior into a supply-chain risk.

Why local auditing matters

A web-based checker asks you to send a relay key to another service before you can test the relay. API Relay Audit avoids that extra trust hop: the script runs locally, and your API key is sent only to the relay URL you specify.

How to audit one

curl -sO https://raw.githubusercontent.com/toby-bridges/api-relay-audit/master/audit.py
python audit.py --key <YOUR_KEY> --url <BASE_URL> --output report.md

What the report does not prove

A clean-looking run is not a certificate. Relays can behave conditionally, models can be ambiguous, and unsupported formats can make a step inconclusive. Treat the report as reproducible evidence, not a final safety label.