---
sidebar_position: 10
---

import Terminal from '@site/src/components/Terminal';

# Observability

Every interaction with a **Large Language Model (LLM)** generates crucial data that can be monitored, analyzed, and optimized.

Our **LLM Gateway** provides **real-time tracking, security, and performance insights**, acting as a **centralized observability layer** to streamline LLM interactions.

By routing all requests through our LLM gateway, you gain:

- **Full-stack observability** by capturing every LLM request before it reaches the provider
- **Granular control** over who can access what, when, and how
- **Cost efficiency** through detailed usage tracking and budgeting
- **Real-time insights** into token consumption, latency, and error rates

## Key metrics tracked

Every LLM request logs critical telemetry data, including:

- **LLM Provider** - Identify the AI service in use (e.g., OpenAI, Anthropic, Mistral)
- **Model Version** - Track which model is processing requests
- **Prompt & Response Data** - Log inputs and outputs for debugging & quality control
- **Token Usage Metrics** - Measure input/output/reasoning token consumption
- **User Identity** - Associate API usage with specific users for accountability
- **API Key Tracking** - Monitor & secure API access to prevent unauthorized use
- **Cost Tracking** - Compute costs per request, per model, per user
- **Ecological Impact** - Estimate energy consumption and CO2 emissions
- **Budget Consumption** - Track spending against defined budget limits

## Using data exporters

You can use [Otoroshi data exporters](https://maif.github.io/otoroshi/manual/entities/data-exporters.html) to extract LLM usage information and send it to any external system (Elasticsearch, Kafka, webhooks, etc.).

![lb](/img/llm-data-exporter.png)

Make sure to filter events on `LLMUsageAudit`:

```json
{
  "include": [{
    "audit": "LLMUsageAudit"
  }],
  "exclude": []
}
```

For detailed information about `LLMUsageAudit` events, event fields, operation types, and dashboard examples, see the [Reporting documentation](/docs/reporting).

## Dashboarding

You can use `LLMUsageAudit` events to build dashboards with tools like Kibana, Grafana, or any visualization tool.

![](/img/grafana-dashboard.png)

See the [Reporting section](/docs/reporting) for Elasticsearch data exporter configuration examples and event structure details.
