# DefenseClaw v7 — Observability Contract

Downstream-facing contract for SIEM, analytics, and integration tests. **Schema version** for JSON envelopes is **7** (`internal/version.SchemaVersion`, `schema_version` on every stamped event). **Generation** is a monotonic counter bumped on config/policy save (`version.BumpGeneration()`); **content_hash** is SHA-256 of canonical JSON config/policy; **binary_version** is the running DefenseClaw semver.

| Source of truth (Go) | Parity (Python / CI) |
|------------------------|----------------------|
| `internal/audit/actions.go` — `AllActions()` | `cli/defenseclaw/audit_actions.py`, `make check-audit-actions` |
| `internal/gatewaylog/error_codes.go` — `AllErrorCodes()`, `AllSubsystems()` | `cli/defenseclaw/gateway_error_codes.py`, `make check-error-codes` |
| `schemas/*.json` | `make check-schemas` |

**Gate:** `make check-v7` before merging changes to actions, error codes, or schemas.

---

## Event types (`gatewaylog.EventType`)

Each emission is one JSON object per line (`gateway.jsonl`) with `event_type` discriminating the payload. Full schema: `schemas/gateway-event-envelope.json` (references `scan-event.json`, `scan-finding-event.json`, `activity-event.json`).

| `event_type` | Payload key | Primary SQLite projection |
|--------------|-------------|---------------------------|
| `verdict` | `verdict` | Mirror `audit_events` when guardrail emits audit twin (`guardrail-*` actions); isolated `gatewaylog.Writer.Emit` tests may not persist |
| `judge` | `judge` | `judge_responses` when retention enabled |
| `lifecycle` | `lifecycle` | Often mirrored from `audit.Logger` via `auditBridge` |
| `error` | `error` | Optional `audit_events` / alerts path depending on subsystem |
| `diagnostic` | `diagnostic` | Operator-opt-in sinks; always stderr-capable |
| `scan` | `scan` | `scan_results` + `audit_events` summary |
| `scan_finding` | `scan_finding` | `scan_findings` |
| `activity` | `activity` | `activity_events` + redacted `audit_events` summary |

### Example: `verdict` (trimmed)

```json
{
  "ts": "2026-04-20T12:00:00Z",
  "event_type": "verdict",
  "severity": "HIGH",
  "schema_version": 7,
  "generation": 1,
  "content_hash": "<sha256>",
  "binary_version": "0.2.0",
  "run_id": "<run>",
  "session_id": "<session>",
  "trace_id": "<32-char-hex>",
  "agent_id": "<logical>",
  "agent_instance_id": "<session-scoped>",
  "sidecar_instance_id": "<process-scoped>",
  "verdict": { "stage": "final", "action": "block", "reason": "…", "latency_ms": 12 }
}
```

---

## Correlation fields

| Field | Role | Source |
|-------|------|--------|
| `run_id` | One per agent invocation | `gatewaylog.SetProcessRunID` at boot (atomic) → `gatewaylog.ProcessRunID()`; falls back to `DEFENSECLAW_RUN_ID` env |
| `session_id` | OpenClaw / conversation key when present | `X-DefenseClaw-Session-Id` header (HTTP surface) or Bifrost stream `session_key` (WebSocket surface) |
| `trace_id` | W3C trace id from OTel span context | `traceparent` header (gateway parses), otherwise minted by plugin per LLM call |
| `request_id` | HTTP request id from proxy / API | `X-DefenseClaw-Request-Id` or generated by `requestIDMiddleware` |
| `agent_id` | Stable logical agent id | `X-DefenseClaw-Agent-Id` header (plugin) or `AgentRegistry.Resolve` |
| `agent_instance_id` | Session-scoped agent instance | Plugin cache or `AgentRegistry.Resolve(ctx, sessionKey, "")` |
| `policy_id` | Active guardrail / admission policy | `X-DefenseClaw-Policy-Id` header or `EventRouter.SetDefaultPolicyID` |
| `destination_app` | Downstream app / tool target | `X-DefenseClaw-Destination-App` header or `toolDestinationApp(provider, qualifier)` on stream events |
| `tool_name`, `tool_id` | Tool call identity | Stamped on emission by `inspectToolCalls` (HTTP proxy) and `router.handleToolCall/Result` (stream) |

**Minting:** Gateway and scanner paths stamp what they know; missing fields stay empty (never fabricated). The correlation envelope (`audit.CorrelationEnvelope`) is the single propagation contract — set by `CorrelationMiddleware` for HTTP handlers and by `EventRouter.streamEnvelope` / `EventRouter.logStreamAction` for Bifrost stream goroutines, consumed by `audit.Logger.LogEventCtx` / `LogActionCtx` / `LogAlertCtx`. Call sites MUST prefer the `*Ctx` variants so runtime fields land in SQLite and sink fan-out; the legacy non-`Ctx` methods are retained only for pre-v7 seed data (boot events before middleware exists).

### Correlation headers (plugin → gateway)

The OpenClaw plugin's fetch interceptor (`extensions/defenseclaw/src/fetch-interceptor.ts`) injects the following headers on every proxied LLM call so `CorrelationMiddleware` can rehydrate the envelope without depending on the request body:

| Header | Value |
|--------|-------|
| `X-DefenseClaw-Agent-Id` | Cached logical agent id |
| `X-DefenseClaw-Agent-Instance-Id` | Cached session-scoped instance id |
| `X-DefenseClaw-Sidecar-Instance-Id` | Sidecar process UUID (echo for three-tier sanity) |
| `X-DefenseClaw-Agent-Name` | Human label |
| `X-DefenseClaw-Session-Id` | From the currently-tracked `before_tool_call` context |
| `X-DefenseClaw-Run-Id` | From the currently-tracked `before_tool_call` context |
| `X-DefenseClaw-Trace-Id` | Minted per intercepted LLM call |
| `X-DefenseClaw-Policy-Id` | Pulled from identity cache |
| `X-DefenseClaw-Destination-App` | Set by caller when known (e.g. `openclaw-ide`) |

Empty values are omitted (never sent as `""`) to keep SQLite rows sparse rather than polluted with stub strings.

---

## Three-tier agent identity

| Field | Meaning | Example |
|-------|---------|---------|
| `agent_id` | Stable logical agent | `openclaw`, plugin id |
| `agent_instance_id` | Session-scoped or process default from `audit.SetProcessAgentInstanceID` | Per conversation or boot UUID |
| `sidecar_instance_id` | **DefenseClaw gateway process** | New UUID each sidecar start |

**Breaking change vs v6:** Do not treat `agent_instance_id` as interchangeable with `sidecar_instance_id`. They answer different questions (who ran vs which binary emitted).

### `agent_id` resolution (operator contract)

`AgentRegistry.Resolve` (`internal/gateway/agent_registry.go`) is intentionally strict about the logical id and only reads from, in order:

1. `X-DefenseClaw-Agent-Id` request header (plugin-supplied per request).
2. `agent.id` in `~/.defenseclaw/config.yaml` (process-wide default).

It does **not** fall back to `agent_name`, the claw-mode string, or stream-provided agent hints. This is by design — `agent_id` is a join key for downstream SIEMs and must be stable across sidecar restarts, which `agent_name` (human label) and `claw.mode` (framework id) are not.

Consequence for operators: if neither source is set, every runtime audit row will carry `agent_id = ""` while still stamping `agent_name`, `agent_instance_id`, and `sidecar_instance_id` normally. To populate `agent_id`:

- **Plugin-scoped (recommended):** set `X-DefenseClaw-Agent-Id` on the fetch interceptor's correlation headers. The OpenClaw plugin already pulls this from its identity cache; verify with `defenseclaw doctor` that `agent.id` is resolved.
- **Sidecar-scoped fallback:** set `agent.id: <your-logical-id>` in `~/.defenseclaw/config.yaml` so every event the sidecar emits (boot, admission, scan) carries the same id even before a plugin request lands.

An empty `agent_id` on runtime rows is a config signal, not a code regression.

---

## Nullability (summary)

Consult `schemas/gateway-event-envelope.json` and nested `$defs`. Envelope fields use `omitempty`; exactly **one** of `verdict` / `judge` / `lifecycle` / `error` / `diagnostic` / `scan` / `scan_finding` / `activity` is populated per `oneOf` in the schema.

---

## Error codes (`gatewaylog.ErrorCode`)

Complete enumeration (must match `AllErrorCodes()`):

- `SINK_DELIVERY_FAILED`, `SINK_QUEUE_FULL`
- `EXPORT_FAILED`
- `CONFIG_LOAD_FAILED`
- `POLICY_LOAD_FAILED`
- `AUTH_INVALID_TOKEN`, `AUTH_MISSING_TOKEN`, `AUTH_CSRF_MISMATCH`, `AUTH_ORIGIN_BLOCKED`
- `INVALID_HEADER`
- `INVALID_RESPONSE`
- `SUBPROCESS_EXIT`
- `WEBHOOK_DELIVERY_FAILED`, `WEBHOOK_COOLDOWN`
- `FS_MOVE_FAILED`, `FS_LINK_FAILED`
- `CLIENT_DISCONNECT`, `UPSTREAM_ERROR`, `STREAM_TIMEOUT`
- `SQLITE_BUSY`
- `PANIC_RECOVERED`
- `LLM_BRIDGE_ERROR`
- `SCHEMA_VIOLATION`

---

## Subsystems (`gatewaylog.Subsystem`)

Complete enumeration (must match `AllSubsystems()`):

`sidecar`, `watcher`, `gateway`, `scanner`, `policy`, `guardrail`, `auth`, `config`, `inspect`, `approvals`, `sink`, `telemetry`, `correlation`, `stream`, `cisco-inspect`, `openshell`, `webhook`, `quarantine`, `agent-registry`, `sqlite`, `admission`, `config_mutation`, `gatewaylog`

---

## Audit actions (`audit.Action`)

Complete enumeration (must match `AllActions()`):

`init`, `stop`, `ready`, `scan`, `scan-start`, `rescan`, `rescan-start`, `block`, `allow`, `warn`, `quarantine`, `restore`, `disable`, `enable`, `deploy`, `drift`, `network-egress-blocked`, `network-egress-allowed`, `guardrail-block`, `guardrail-warn`, `guardrail-allow`, `approval-request`, `approval-granted`, `approval-denied`, `tool-call`, `tool-result`, `config-update`, `policy-update`, `policy-reload`, `action`, `acknowledge-alerts`, `dismiss-alerts`, `webhook-delivered`, `webhook-failed`, `sink-failure`, `sink-restored`, `alert`, `otel.ingest.logs`, `otel.ingest.metrics`, `otel.ingest.traces`, `otel.ingest.malformed`, `connector-hook`, `asset-policy`, `codex.notify`, `codex.notify.agent-turn-complete`, `codex.notify.malformed`

---

## Surface matrix (five observability tiers)

| Tier | Mechanism |
|------|-----------|
| 1 — SQLite | `audit_events`, `activity_events`, `scan_results` / `scan_findings`, `judge_responses`, … |
| 2 — Gateway JSONL | `gatewaylog.Writer` → file + fanout |
| 3 — OTel traces | `sdktrace` spans (proxy, tools, approvals, …) |
| 4 — OTel metrics | `sdkmetric` instruments (`defenseclaw.*`, `gen_ai.*`, …) |
| 5 — Audit sinks | `audit_sinks` → Splunk HEC, OTLP logs, HTTP JSONL (`sinks.Event`; wire shape mirrors `audit.Event` with optional `structured`) |

Not every `event_type` duplicates into all five tiers; gateway-native emissions may skip SQLite until a mirror row exists.

---

## Breaking changes vs v6

- **Identity:** three-tier model; column / field rename: prefer `sidecar_instance_id` for process scope.
- **Schema:** `schema_version == 7` minimum for provenance fields on audited rows and gateway envelopes.

---

## Downstream TODOs

- TODO: Reject or quarantine events with `schema_version` < 7 once fleet is migrated.
- TODO: Dashboards: bucket by `content_hash` + `generation` after policy reloads.
- TODO: Join `scan.scan_id` → `scan_findings` → `audit_events` via `run_id` / `trace_id`.
- TODO: `sinks.Event` forward path may omit zero `schema_version` in JSON — validate against persisted `audit_events` when strict schema compliance is required (see `TODO(v7-followup)` in `test/e2e/v7_observability_test.go`).

---

*Validated by `go test ./test/e2e/ -run TestObservabilityContractDocListsMatchGo`*
