# Audit Ledger

> **Status:** Available since v6.0.0
> **EU AI Act:** Articles 17/19 — Immutable Decision Traceability

The Audit Ledger provides a **write-once, append-only** trail of every lifecycle task execution. It captures what data each task read, what it produced, LLM-specific details (compiled prompts, model responses, token usage), tool calls, actions, costs, and timing — signed with HMAC-SHA256 for tamper detection.

## Overview

Every time a conversation turn is processed, each lifecycle task (parser, behavior rules, HTTP calls, LangChain, output, etc.) generates an audit entry. These entries are:

1. **Scrubbed** — secrets are redacted (API keys, bearer tokens, vault references)
2. **Signed** — HMAC-SHA256 computed over all fields for tamper detection
3. **Batched** — queued in-memory and flushed to the database every few seconds
4. **Immutable** — stored in a write-once collection with no update or delete operations

## Configuration

| Property                            | Default | Description                                                 |
| ----------------------------------- | ------- | ----------------------------------------------------------- |
| `eddi.audit.enabled`                | `true`  | Enable/disable the audit ledger                             |
| `eddi.audit.flush-interval-seconds` | `3`     | How often to flush queued entries to the database           |
| `EDDI_VAULT_MASTER_KEY`             | (none)  | Vault master key — also used to derive the HMAC signing key |

> **Note:** If `EDDI_VAULT_MASTER_KEY` is not set, audit entries are stored without HMAC integrity hashes. A warning is logged at startup.

## Audit Entry Structure

Each audit entry captures:

| Field            | Type    | Description                                                |
| ---------------- | ------- | ---------------------------------------------------------- |
| `id`             | UUID    | Auto-generated unique identifier                           |
| `conversationId` | String  | Conversation this entry belongs to                         |
| `agentId`        | String  | Agent identifier                                           |
| `agentVersion`   | Integer | Agent version                                              |
| `userId`         | String  | User identifier                                            |
| `environment`    | String  | Deployment environment (e.g., `production`)                |
| `stepIndex`      | int     | 0-based step position in the conversation                  |
| `taskId`         | String  | Lifecycle task ID (e.g., `ai.labs.parser`)                 |
| `taskType`       | String  | Task type (e.g., `expressions`, `langchain`)               |
| `taskIndex`      | int     | 0-based task position in the pipeline                      |
| `durationMs`     | long    | Task execution time in milliseconds                        |
| `input`          | Map     | Data read by the task (user input, actions)                |
| `output`         | Map     | Data written by the task (output text, tool results)       |
| `llmDetail`      | Map     | LLM-specific: compiled prompt, model response, token usage |
| `toolCalls`      | Map     | Tool execution: name, args, result, cost                   |
| `actions`        | List    | Actions emitted by this task                               |
| `cost`           | double  | Monetary cost of this step                                 |
| `timestamp`      | Instant | When the task completed                                    |
| `hmac`           | String  | HMAC-SHA256 integrity hash                                 |

## REST API

The audit ledger exposes a **read-only** REST API. No create, update, or delete endpoints exist.

### Get Audit Trail by Conversation

```
GET /auditstore/{conversationId}?skip=0&limit=100
```

Returns audit entries for a conversation, newest first.

### Get Audit Trail by Agent

```
GET /auditstore/agent/{agentId}?agentVersion=1&skip=0&limit=100
```

Returns audit entries for an agent. The `agentVersion` parameter is optional.

### Get Entry Count

```
GET /auditstore/{conversationId}/count
```

Returns the total number of audit entries for a conversation.

## HMAC Integrity

When the vault master key is configured, each audit entry is signed with HMAC-SHA256:

1. A **signing key** is derived from the vault master key using PBKDF2 with a distinct salt (`eddi-audit-hmac-v1`, 600K iterations). This makes the audit signing key cryptographically independent from the vault's KEK.
2. A **canonical string** is built from all entry fields (excluding the HMAC itself), with map keys sorted alphabetically for deterministic output.
3. The HMAC is computed and stored as a hex-encoded string.

To verify an entry has not been tampered with, recompute the HMAC and compare it to the stored value.

## Secret Redaction

All string values in audit entries pass through the `SecretRedactionFilter` before storage. The following patterns are redacted:

- OpenAI API keys (`sk-...`)
- Anthropic API keys (`sk-ant-...`)
- Bearer tokens (JWTs and opaque tokens)
- Generic API key patterns (`apikey=...`, `token=...`, etc.)
- Vault references (`${vault:...}`)

Redaction is applied recursively to nested maps and lists.

## Failure Handling

If a database write fails, entries are **re-queued** for the next flush cycle. After 3 consecutive failures, entries are dropped and an error is logged. This prevents unbounded memory growth while maximizing data retention.

## Storage

### MongoDB (default)

- Collection: `audit_ledger`
- Indexes: `conversationId`, `(agentId, agentVersion)`, `timestamp` (descending)
- Operations: `insertOne`, `insertMany` only — no update or delete

### PostgreSQL

- Table: `audit_ledger` (auto-created on first use)
- Hybrid storage: indexed columns (conversation_id, agent_id, agent_version, timestamp) + JSONB for variable data
- Activated with `@IfBuildProfile("postgres")`
- Same insert-only contract as MongoDB

## Architecture

```
LifecycleManager                 ConversationService
  |                                |
  | buildAuditEntry()              | setAuditCollector()
  | (per task completion)          | (enriches with environment)
  |                                |
  v                                v
IAuditEntryCollector ---------> AuditLedgerService
                                   |
                                   | 1. scrubSecrets()
                                   | 2. computeHmac()
                                   | 3. queue.offer()
                                   |
                                   v  (every N seconds)
                                IAuditStore.appendBatch()
                                   |
                          +--------+--------+
                          |                 |
                     AuditStore     PostgresAuditStore
                     (MongoDB)        (PostgreSQL)
```
