---
sidebar_position: 1
---

import Terminal from '@site/src/components/Terminal';

# Reporting

You can use the audit events generated by your LLM usage to make some reporting dashboard and follow your metrics in near real-time.

Every interaction with an LLM provider generates a `LLMUsageAudit` event that contains detailed information about the request, the response, the provider, the costs, the ecological impacts, the budgets, and more.

## LLMUsageAudit event fields

| Field | Type | Description |
|-------|------|-------------|
| `@id` | string | Unique event identifier |
| `@timestamp` | number | Event timestamp in milliseconds |
| `@type` | string | Always `"AuditEvent"` |
| `@product` | string | Always `"otoroshi"` |
| `audit` | string | Always `"LLMUsageAudit"` |
| `provider_kind` | string | The LLM provider type (e.g., `openai`, `anthropic`, `mistral`, etc.) |
| `provider` | string | The provider entity ID |
| `duration` | number | Request duration in milliseconds |
| `model` | string | The model used for the request |
| `rate_limit` | object | Rate limit information from the provider (`requests_limit`, `requests_remaining`, `tokens_limit`, `tokens_remaining`) |
| `usage` | object | Token usage (`prompt_tokens`, `generation_tokens`, `reasoning_tokens`) |
| `error` | object/null | Error details if the request failed, `null` on success |
| `consumed_using` | string | The type of operation (see below) |
| `user` | object/null | The authenticated user, if any |
| `apikey` | object/null | The API key used, if any |
| `route` | object/null | The Otoroshi route that handled the request |
| `input_prompt` | array/object | The input prompt or request body |
| `output` | object/array | The LLM response |
| `provider_details` | object | Full provider configuration |
| `impacts` | object/null | Ecological impact data (if [enabled](/docs/reporting/ecological)) |
| `costs` | object/null | Cost tracking data (if [enabled](/docs/cost-optimizations/cost-tracking)) |
| `budgets` | object/null | Budget consumption data (if [enabled](/docs/cost-optimizations/budgets)) |
| `consumer_rate_limit` | object/null | Consumer token rate limit information |

## Operation types (`consumed_using`)

The `consumed_using` field indicates what kind of LLM operation generated the event:

| Value | Description |
|-------|-------------|
| `chat/completion/blocking` | Synchronous chat completion |
| `chat/completion/streaming` | Streaming chat completion |
| `completion/blocking` | Synchronous text completion |
| `completion/streaming` | Streaming text completion |
| `embedding_model/embedding` | Text embedding generation |
| `audio_model/translate` | Audio translation |
| `audio_model/stt` | Speech-to-text transcription |
| `image_model/generate` | Image generation |
| `image_model/edit` | Image editing |
| `moderation_model/moderate` | Content moderation |
| `video_model/generate` | Video generation |

## Dashboard example

In the following screenshot, we used an [Elasticsearch data exporter](https://maif.github.io/otoroshi/manual/entities/data-exporters.html#elastic) to send
`LLMUsageAudit` events to an Elasticsearch instance and Kibana to create a dashboard based on those events.

![](/img/kibana.png)

## Elastic data exporter example

Go to [Data Exporters](http://otoroshi.oto.tools:8080/bo/dashboard/exporters) page and create a new one.

```js
{
  "_loc": {
    "tenant": "default",
    "teams": [
      "default"
    ]
  },
  "type": "elastic",
  "enabled": true,
  "id": "data_exporter_be7a6e21-152b-4c09-b384-15f9dbb8041f",
  "name": "New elastic exporter config",
  "desc": "New elastic exporter config",
  "metadata": {},
  "tags": [],
  "bufferSize": 5000,
  "jsonWorkers": 1,
  "sendWorkers": 5,
  "groupSize": 100,
  "groupDuration": 30000,
  "projection": {},
  "filtering": {
    "include": [
      {
        "audit": "LLMUsageAudit"
      }
    ],
    "exclude": []
  },
  "config": {
    "clusterUri": "http://localhost:9200",
    "uris": [
      "http://localhost:9200"
    ],
    "index": "otoroshi-llm",
    "type": null,
    "user": null,
    "password": null,
    "headers": {},
    "indexSettings": {
      "clientSide": true,
      "interval": "Day",
      "numberOfShards": 1,
      "numberOfReplicas": 1
    },
    "mtlsConfig": {
      "certs": [],
      "trustedCerts": [],
      "mtls": false,
      "loose": false,
      "trustAll": false
    },
    "applyTemplate": true,
    "version": null,
    "maxBulkSize": 100,
    "sendWorkers": 4
  },
  "kind": "events.otoroshi.io/DataExporter"
}
```

## Extra analytics data in GatewayEvent

In addition to the dedicated `LLMUsageAudit` events, the LLM extension also enriches the standard Otoroshi `GatewayEvent` (the analytics event generated for every HTTP request passing through a route) with LLM-specific data. This data is added to the `ExtraAnalyticsData` field of the `GatewayEvent` using different keys depending on the operation type.

### Analytics data keys

| Key | Operation type | Added by |
|-----|----------------|----------|
| `ai` | Chat completion, text completion | LLM providers (OpenAI, Anthropic, Mistral, etc.) |
| `ai-embedding` | Embedding generation | Embedding model auditing decorator |
| `ai-audio` | Audio translation, speech-to-text | Audio model auditing decorator |
| `ai-image` | Image generation, image editing | Image model auditing decorator |
| `ai-moderation` | Content moderation | Moderation model auditing decorator |
| `ai-video` | Video generation | Video model auditing decorator |
| `ai-consumer-rate-limit` | Token rate limiting | [LLM Token Rate Limiting](/docs/cost-optimizations/quotas) plugin |

Each key contains an array of objects, one per LLM call made during the request processing. This allows tracking multiple LLM calls within a single HTTP request (e.g., when using fallbacks or load balancing).

### `ai` key structure (chat/completion)

For chat and text completion operations, each provider adds a slug to the `ai` array with the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `provider_kind` | string | The provider type (e.g., `"openai"`, `"AzureOpenAi"`, `"Anthropic"`) |
| `provider` | string | The provider entity ID |
| `duration` | number | Request duration in milliseconds |
| `model` | string | The model used |
| `rate_limit` | object | Rate limit information (`requests_limit`, `requests_remaining`, `tokens_limit`, `tokens_remaining`) |
| `usage` | object | Token usage (`prompt_tokens`, `generation_tokens`, `reasoning_tokens`) |
| `cache` | object | Cache information (if a cache hit occurred) |
| `deployment_id` | string | Azure OpenAI deployment ID (Azure providers only) |
| `resource_name` | string | Azure OpenAI resource name (Azure providers only) |

### `ai-consumer-rate-limit` key structure

The [LLM Token Rate Limiting](/docs/cost-optimizations/quotas) plugin adds an `ai-consumer-rate-limit` array to the extra analytics data. Each entry in the array is appended during response or error transformation, with the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `max_tokens` | number | Maximum allowed tokens in the window |
| `window_millis` | number | Rate limit window duration in milliseconds |
| `consumed_tokens` | number | Tokens consumed in the current window |
| `remaining_tokens` | number | Remaining tokens in the current window |

### `ai-embedding` key structure

| Field | Type | Description |
|-------|------|-------------|
| `provider_kind` | string | The provider type |
| `provider` | string | The provider entity ID |
| `duration` | number | Request duration in milliseconds |
| + response fields | | The full embedding response (model, usage, data) |

### `ai-audio` key structure

| Field | Type | Description |
|-------|------|-------------|
| `provider_kind` | string | The provider type |
| `provider` | string | The provider entity ID |
| `duration` | number | Request duration in milliseconds |
| + response fields | | The full audio response (text, language, duration, segments) |

### `ai-image` key structure

| Field | Type | Description |
|-------|------|-------------|
| `provider_kind` | string | The provider type |
| `provider` | string | The provider entity ID |
| `duration` | number | Request duration in milliseconds |
| + response fields | | The full image response (created, data) |

### `ai-moderation` key structure

| Field | Type | Description |
|-------|------|-------------|
| `provider_kind` | string | The provider type |
| `provider` | string | The provider entity ID |
| `duration` | number | Request duration in milliseconds |
| + response fields | | The full moderation response (id, model, results) |

### `ai-video` key structure

| Field | Type | Description |
|-------|------|-------------|
| `provider_kind` | string | The provider type |
| `provider` | string | The provider entity ID |
| `duration` | number | Request duration in milliseconds |
| + response fields | | The full video response |

### Example GatewayEvent extra analytics data

```json
{
  "ai": [
    {
      "provider_kind": "openai",
      "provider": "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
      "duration": 415,
      "model": "gpt-4o-mini",
      "rate_limit": {
        "requests_limit": 10000,
        "requests_remaining": 9999,
        "tokens_limit": 200000,
        "tokens_remaining": 199993
      },
      "usage": {
        "prompt_tokens": 11,
        "generation_tokens": 18,
        "reasoning_tokens": 0
      }
    }
  ],
  "ai-consumer-rate-limit": [
    {
      "max_tokens": 1000,
      "window_millis": 60000,
      "consumed_tokens": 29,
      "remaining_tokens": 971
    }
  ]
}
```

## LLMUsageAudit event example

Here is a full example of a `LLMUsageAudit` event for a chat completion request:

```json
{
  "@id": "1905616593920983819",
  "@timestamp": 1743169375292,
  "@type": "AuditEvent",
  "@product": "otoroshi",
  "@serviceId": "",
  "@service": "Otoroshi",
  "@env": "dev",
  "audit": "LLMUsageAudit",
  "provider_kind": "openai",
  "provider": "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
  "duration": 415,
  "model": "gpt-4o-mini",
  "rate_limit": {
    "requests_limit": 10000,
    "requests_remaining": 9999,
    "tokens_limit": 200000,
    "tokens_remaining": 199993
  },
  "usage": {
    "prompt_tokens": 11,
    "generation_tokens": 18,
    "reasoning_tokens": 0
  },
  "error": null,
  "consumed_using": "chat/completion/blocking",
  "user": null,
  "apikey": null,
  "route": {
    "id": "route_e4a9d6cb3-d859-4203-a860-8d1dd6d09557",
    "name": "test",
    "..."
  },
  "input_prompt": [
    {
      "role": "user",
      "content": "tell me a joke"
    }
  ],
  "output": {
    "generations": [
      {
        "message": {
          "role": "assistant",
          "content": "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
        }
      }
    ],
    "metadata": {
      "rate_limit": { "..." },
      "usage": { "..." },
      "costs": {
        "input_cost": 0.00000165,
        "output_cost": 0.0000108,
        "reasoning_cost": 0,
        "total_cost": 0.00001245,
        "currency": "dollar"
      }
    }
  },
  "provider_details": {
    "id": "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
    "name": "OpenAI",
    "provider": "openai",
    "..."
  },
  "impacts": null,
  "costs": {
    "input_cost": 0.00000165,
    "output_cost": 0.0000108,
    "reasoning_cost": 0,
    "total_cost": 0.00001245,
    "currency": "dollar"
  },
  "budgets": null,
  "consumer_rate_limit": null
}
```
