# ADR-0042: Local Append-Only Invocation and HTTP Request Log

## Status

✅ Accepted (2026-07-07)

## Context

`omni-dev` had exactly one observability surface: `RUST_LOG` tracing to stderr.
That is **ephemeral** (gone when the terminal scrolls or the process exits),
**unstructured** (free-form log lines), and **stderr-only** (nothing durable to
query later). It answers "what is happening right now while I watch," not "what
did this tool run last Tuesday, and which of those runs hit a 5xx from JIRA."

As the tool grew — multiple AI backends, Atlassian/Datadog/Snowflake REST
clients, the browser bridge, the MCP server, the daemon — the questions that
mattered became historical and cross-invocation:

- *Which command made that request, and did it retry?*
- *Which runs failed last week, and with what error?*
- *What did the daemon do on behalf of which CLI invocation?*

None of these are answerable from ephemeral stderr tracing. They need a durable,
structured, queryable record that survives the process and can be filtered after
the fact. This is a distinct architectural surface, not a tracing tweak, and it
was shipped without an ADR across #1121, #1122, #1129, #1133, and #1134. This
record captures *why* it is shaped the way it is.

The constraints that shaped the design:

- It is **infrastructure that wraps every command and every HTTP client**, so it
  must never change a caller's behaviour or exit code — an observer that can
  break the observed is worse than no observer.
- Files written today are read by binaries built months from now (and vice
  versa), so the on-disk shape is a **wire contract** that is expensive to
  change once logs exist on disk.
- The two things worth recording — *what was run* and *what it talked to* — are
  produced at very different layers (the `main.rs`/MCP shell versus deep inside
  each client's retry loop), yet must be **correlated** without threading a
  logging handle through every intervening call site. The MCP server makes this
  acute: it multiplexes many tool calls through one process, so a process-global
  "current invocation" is not enough.
- It is **local-machine state** holding a history of what you ran and where you
  connected, so it must carry no secrets and sit behind the same filesystem
  posture as other `omni-dev` runtime state.

## Decision

Add a **local, append-only NDJSON log** (`log.jsonl`) recording one record per
invocation and one per outbound HTTP request, correlated by a shared
`invocation_id`, plus an `omni-dev log` subcommand to search and pretty-print it.
The engine is [`src/request_log.rs`](../../src/request_log.rs); the query/prune
CLI is [`src/cli/log.rs`](../../src/cli/log.rs) and
[`src/cli/log/{format,query,stream,prune}.rs`](../../src/cli/log/). The
operator-facing guide is [docs/log.md](../log.md).

### A persistent, queryable log — positioned against ephemeral tracing

The log is deliberately **not** a second tracing sink. It is a product surface
with its own command. One JSON object per line (NDJSON), with a `kind` field
discriminating two record types in a single stream:

- **`kind: "invocation"`** — one per process run *and* one per MCP tool call:
  the resolved subcommand path, full (scrubbed) argv, exit code, wall-clock
  duration, any top-level error, and a whitelisted `OMNI_DEV_*` env snapshot.
- **`kind: "http"`** — one per outbound request, recorded *inside* each client's
  retry loop so retries and transport failures are captured too: service tag,
  method, URL, status, elapsed, and any error.

NDJSON was chosen because it is append-friendly (a write is a single `O_APPEND`
line), survivable (a torn tail line loses one record, not the file), and
composes with the ecosystem (`omni-dev log -o json | jq …`). The `omni-dev log`
reader adds a filter matrix, a `--query` mini-language (AND/OR/NOT, `field:value`,
numeric comparators, fuzzy tokens), three renderers (`oneline`/`json`/`full`),
and `-f/--follow`. `tracing` remains the right tool for live, verbose,
per-line debugging; the log is the right tool for durable, structured,
after-the-fact questions. They coexist rather than compete.

### Best effort: logging can never fail the caller

`record()` is the single choke point, and it **swallows every error** — a failed
path resolution, a serialize error, a full disk, a permission problem — logging
the failure only at `tracing::debug`. A logging fault can therefore never change
the command's exit code or output. `OMNI_DEV_LOG_DISABLE=1` is an absolute
opt-out that short-circuits before any work.

This is a real, deliberate tradeoff: it prefers **caller safety over record
completeness**. A log write can be silently lost and the caller will never know.
The alternative — surfacing logging errors — was rejected because it would let an
observability feature break the thing it observes, which is unacceptable for
infrastructure that wraps *every* command. The log is an aid, not a
system-of-record with delivery guarantees, and it is shaped to say so.

### One forward-compatible `LogRecord` for both read and write

A single [`LogRecord`](../../src/request_log.rs) struct is used for **both**
writing and reading. Every field is `#[serde(default)]` (a missing field reads as
its default rather than failing), and every optional field is
`skip_serializing_if` (absent fields never hit the wire, keeping lines compact).
The two enums (`RecordKind`, `Source`) carry a `#[serde(other)] Unknown` variant,
so a kind or source written by a newer binary deserializes rather than erroring.

The consequence is a **forward-rolling contract**: a newer reader never chokes on
an older line, and an older reader never chokes on a newer one — the same
discipline the daemon wire types use ([ADR-0039](adr-0039.md)). This is chosen
precisely because it is *expensive to change once files exist on disk*: there is
no schema migration for an append-only file that older binaries still read, so
the schema is designed to only ever grow, never break. Record ids are
time-sortable (`{millis:013}-{rand:016x}`), so sorting lines by `id` approximates
sorting by time without parsing timestamps.

### Context propagation: `OnceLock` (process) + `tokio::task_local!` (task)

An HTTP record is emitted deep inside a client's retry loop; it must learn its
parent `invocation_id`/`source`/`mcp_tool` without every client signature growing
a logging parameter. The `RequestLogContext` solves this with a two-tier resolver
([`current_context()`](../../src/request_log.rs)):

1. A process-global `OnceLock<RequestLogContext>` set **once, very early**, by the
   `main.rs` shell (and by `daemon run`). This covers the common single-invocation
   process.
2. A `tokio::task_local!` override (`CTX`) scoped around each unit of work that
   needs its *own* identity within a shared process. The MCP server sets it per
   tool call (`RequestLogContext::mcp(tool)`), because one MCP process multiplexes
   many logically distinct invocations — a process-global would collapse them all
   into one `invocation_id`.

`current_context()` reads the task-local first, then the global, then synthesizes
a fallback, so any HTTP hook — in any client, on any task — resolves the right
parent with zero threading. The daemon and browser bridge extend this with
`scope_origin_id()`: a request served *on behalf of* a CLI/MCP client is scoped to
the **caller's** `invocation_id` (threaded across the control socket for Snowflake
and the `X-Omni-Bridge-Origin` header for the bridge), so `omni-dev log --id
<cli-invocation>` surfaces both halves of a daemon-served operation while `source`
stays `daemon` and `via_daemon` remains true (#1198).

### Growth bounds are opt-in only

The log is **default-on** for every invocation and every request, so on an active
machine it grows steadily and without bound unless the operator intervenes. Two
bounds exist, and **both are off by default**:

- **Rotation on write** (unix-only), gated on `OMNI_DEV_LOG_MAX_SIZE`
  (+ `OMNI_DEV_LOG_KEEP_FILES`, default 3): a stat-rotate-append sequence
  serialized on a stable `log.jsonl.lock` file, producing numbered `log.jsonl.1`,
  `.2`, … files and deleting the oldest beyond the keep count.
- **`omni-dev log prune`** (`--older-than` / `--max-size` / `--dry-run`): an
  age-then-size filter that rewrites the file via a same-directory temp file and
  atomic rename, so a concurrent reader never sees a half-written file.

Defaulting these **off** is deliberate and debatable: it means the log grows until
the operator opts into a bound, but the alternative — silently deleting records
by default — would quietly discard an audit trail the operator may be relying on.
The design prefers *unbounded-but-complete* over *bounded-but-silently-lossy*, and
makes the bound an explicit, discoverable choice (documented in
[docs/log.md](../log.md)).

### Filesystem posture reused from the daemon

Paths resolve as `OMNI_DEV_LOG_FILE` override → `dirs::state_dir()` →
`dirs::data_dir()`, joined with `omni-dev/log.jsonl`. The parent directory is
created `0700` and the file `0600`, reusing `daemon::paths::ensure_dir_0700` /
`ensure_handle_0600` so the log sits behind the **same local-user filesystem
posture** as the daemon socket and token. The file is re-tightened to `0600` via
its handle on every open (no path race), so a pre-existing looser-perm file — an
older version's, or a user-chosen `OMNI_DEV_LOG_FILE` target — is corrected on the
next write (#1139). An existing directory is never re-`chmod`ed (a shared temp dir
or a user-chosen location must not be tightened underneath the user).

### Secrets never reach the log

The log records URLs, argv, and env, so redaction is **core, not an add-on**. No
secret material is written under any code path: auth headers/tokens are redacted
centrally (only a non-secret `auth_principal` identity is kept), secret-bearing
URL query/fragment parameter *values* are redacted while keys are preserved (so
`--url` filtering still works), argv is scrubbed in both `--flag value` and
`--flag=value` forms, and the `OMNI_DEV_*` env snapshot redacts secret-looking
names. Request/response bodies and headers are **opt-in** only
(`OMNI_DEV_LOG_BODIES=1` / `OMNI_DEV_LOG_HEADERS=1`). The exact matchers are in
[docs/log.md](../log.md#redaction-posture).

The *centralized secret-redaction policy* itself — the
[`Secret`](../../src/utils/secret.rs) newtype and the matcher lists that reach
beyond the log into the Atlassian/Datadog/Snowflake clients — is a cross-cutting
concern proposed for its **own**, separate ADR, since it is not specific to the
log. This ADR records only that the log *depends on* that policy and adds its own
URL/argv/env scrubbing on top.

## Consequences

- **The tool gains a durable memory.** "What did I run, when, against what, and
  did it fail?" is now answerable after the fact with one command, across CLI,
  MCP, and daemon-served work — something ephemeral stderr tracing never
  provided.
- **Coverage is in-process only.** Only requests issued by `omni-dev`'s own HTTP
  clients are recorded. GitHub operations that shell out to the `gh` CLI run in a
  separate process and are invisible to the log — the parent invocation is
  recorded, but the network calls `gh` makes are not. This is an accepted
  boundary of the in-process hook approach.
- **The schema can only grow.** New fields must be additive (`#[serde(default)]` +
  `skip_serializing_if`) and new kinds/sources must fall back to `Unknown`;
  renaming or repurposing a field breaks older readers still parsing the same
  file. The forward-compatible struct makes the safe path the easy path, but the
  discipline is now load-bearing.
- **Records can be silently lost.** Best-effort logging means a full disk, a
  permission fault, or a torn write drops a record with no signal beyond a
  `tracing::debug` line. This is intentional (caller safety wins), but it means
  the log is not a delivery-guaranteed audit trail.
- **Unbounded by default.** Left alone, the log grows forever. Operators who care
  must opt into rotation or `prune`; the tradeoff buys a complete-by-default
  history at the cost of disk the operator must reclaim deliberately.
- **No new trust boundary.** The log is local-machine state with the same
  `0700`/`0600` posture as other runtime state ([ADR-0039](adr-0039.md)); it
  opens no socket and persists no secret. Anything that can read the file can see
  your command history and the (non-secret) endpoints you contacted — but reading
  it already requires being the owning local user.
- **Correlation is free at the call site.** Because context resolves through the
  `OnceLock`/task-local pair, adding a new HTTP client or a new invocation surface
  needs one `record_http*` hook and no plumbing; the parent invocation is found
  automatically. The cost is a small amount of hidden global/task-local state that
  a reader must know about to understand how an HTTP record learns its
  `invocation_id`.

See [ADR-0039](adr-0039.md) for the daemon paths/wire-contract conventions this
reuses, [ADR-0036](adr-0036.md) for the browser bridge whose daemon-served
requests this correlates, and [docs/log.md](../log.md) for the operator-facing
guide, the full `--query` grammar, and the exact redaction matchers.
