% RAB v0.1.1 — arXiv preprint
% Minimal arXiv-friendly skeleton: article class + booktabs + hyperref.
% Build:  pdflatex main && bibtex main && pdflatex main && pdflatex main
\documentclass[11pt,a4paper]{article}
\usepackage[margin=1in]{geometry}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{microtype}
\usepackage{booktabs}
\usepackage{array}
\usepackage{enumitem}
\usepackage{hyperref}
\usepackage{url}
\usepackage{xcolor}
\usepackage{listings}
\usepackage{amsmath}
\usepackage{caption}
\hypersetup{
  colorlinks=true,
  linkcolor=blue!50!black,
  citecolor=blue!50!black,
  urlcolor=blue!50!black,
  pdftitle={RAB: A Replayable-Audit Benchmark for RAG and Agent Systems},
  pdfauthor={Jiwon Seo},
  pdfkeywords={EU AI Act, audit log, event sourcing, RAG, agent benchmark, replay}
}
\lstset{
  basicstyle=\ttfamily\small,
  breaklines=true,
  columns=fullflexible,
  frame=single,
  framerule=0.3pt,
  xleftmargin=4pt,
  xrightmargin=4pt
}

\title{RAB: A Replayable-Audit Benchmark for RAG and Agent Systems Operationalising EU AI Act Articles 10, 12, 19\thanks{Pre-registrations locked before any measurement: scenario-S1 measurement protocol at \texttt{docs/research/r1-4-preregistration-2026-06-10.md} (commit \texttt{8a35dee}); scenario-S2 design and reporting protocol at \texttt{docs/research/r1-phase-3-scenario-s2-preregistration-2026-06-10.md} (commit \texttt{d21c680}; archived independently at Zenodo DOI \href{https://doi.org/10.5281/zenodo.20635130}{10.5281/zenodo.20635130}). All references in the JAMES repository.}}

\author{
  Jiwon Seo\thanks{Hashevolution, Republic of Korea. ORCID 0009-0002-0007-7860. Correspondence: \texttt{karu-7@hanmail.net}.}
}

\date{2026-06-11}

\begin{document}

\maketitle

\begin{abstract}
The EU AI Act's high-risk-system obligations apply from 2~August~2026 (Article 113), requiring such systems to keep automatic event logs (Art.~12), trace data origins (Art.~10), and retain logs for at least six months (Art.~19). An open survey on agentic systems~\cite{wang2026evidencetrace} explicitly names \emph{realistic execution-trace benchmarks} as a missing piece of the audit ecosystem. Independently, recent runtimes such as ActiveGraph~\cite{nakajima2026activegraph} and JAMES demonstrate that event-sourced logs with deterministic replay are constructible; what has been missing is the measurement.

We present \textbf{RAB} (Replayable-Audit Benchmark) v0.1.1: a frozen benchmark specification, a deterministic scorer, \emph{two} pre-registered lifecycle scenarios at different scales, and an adapter contract through which any RAG or agent system that can export an append-only audit log is scoreable. RAB defines three metrics --- \textbf{AC} (Audit Completeness), \textbf{RF} (Replay Fidelity, with a separate cost column), and \textbf{PC} (Provenance Coverage) --- each anchored verbatim to EU AI Act provisions and the W3C PROV vocabulary, each computed without invoking any language model. To our knowledge, RAB is the first benchmark that \emph{scores the exported audit-log artifact} (completeness / log-only replay / provenance coverage) of arbitrary RAG and agent systems against EU AI Act record-keeping articles 10/12/19. Behavioural-determinism harnesses such as DFAH~\cite{khatchadourian2026dfah} measure a complementary axis --- \emph{what the agent outputs} --- and AI-Act benchmarking suites such as COMPL-AI~\cite{guldimann2024complai} operationalise the AI Act at the \emph{model} level; RAB occupies the system-level record-keeping axis these two leave open. The headline of a RAB release is a \emph{gap table} across system-under-test categories; no single system's score is the headline. We measure four SUTs on two scenarios. On scenario-S1 (40~ops): a self-verifying reference adapter (AC/RF/PC $=1.0$, $1.0$, $1.0$), the JAMES audit-native runtime ($1.0$, $1.0$, $1.0$), a bolt-on OpenTelemetry-GenAI tracing baseline ($0.5$, $0$, $0$; 49 emitted citations, 0 traceable), and a vanilla quickstart + default-logging floor ($0.275$, $0$, $0$). The four-row gap reveals \emph{two distinct fail-modes} of non-audit-native instrumentation: default logging catches AC INGEST but misses ANSWER, and bolt-on tracing catches AC ANSWER but misses INGEST --- they fail at different canonical categories, and both fail at RF and PC entirely. On scenario-S2 (400~ops, ten times the size, with longer supersede chains and denser cross-references), every cell of the gap table reproduces to the third decimal place: same direction, same per-canonical structure, same disjoint fail-modes; this confirms the gap structure is not a function of corpus size at the order of magnitude tested. A separately pre-registered hypothesis on the RF-cost axis (super-linear scaling of log-only replay cost with corpus size) is not confirmed at this scale and the corresponding claim is withheld per the pre-registration's locked decision rule. The single-number AC overall is therefore explicitly not the headline; the per-canonical-type and per-axis decomposition is. RAB does not certify compliance, does not claim the audit-native architecture is novel, and does not present JAMES's score as a finding. The benchmark, with its honesty clauses and two independently pre-registered measurement protocols, is the contribution. All artefacts (spec, both scenarios, scorer, four adapters, eight result triples across two scenarios, OpenTelemetry-GenAI mapping table) are released under Zenodo DOI~\href{https://doi.org/10.5281/zenodo.20625533}{10.5281/zenodo.20625533} (v0.4.3 software archive) with the scenario-S2 pre-registration separately archived at Zenodo DOI~\href{https://doi.org/10.5281/zenodo.20635130}{10.5281/zenodo.20635130}; both are reproducible bit-for-bit from the JAMES repository (tag \texttt{v0.4.3} for the software archive; commit \texttt{753022b} for the scenario-S2 measurement).
\end{abstract}

\section{Introduction}
\label{sec:intro}

Article~12 of Regulation~(EU)~2024/1689~\cite{aiact2024} requires that high-risk AI systems \emph{``allow for the automatic recording of events (logs) over the lifetime of the system''} that are \emph{``traceable in terms of the functioning of the system''}, with logs facilitating ex-post monitoring (Art.~12(2)(b)) and retained at least six months (Art.~19). Article~10(2)(b) requires that data preparation operations record \emph{``origin of data''}. These requirements apply from 2~August~2026 (Art.~113). Operators of RAG-based assistants, agentic systems, and decision-support tools therefore face a measurable obligation: can their system, given only the exported log, reconstruct what was indexed, what was retrieved, what was synthesised, and what was answered?

The relevant runtimes exist. Event sourcing~\cite{eventsourcing} as a design pattern is decades old, and recent agent runtimes have ported it: ActiveGraph~\cite{nakajima2026activegraph} centers an append-only event log as the source of truth and projects a graph from it; JAMES~\cite{seo2026jamesv043} ships an \texttt{audit\_log} schema with deterministic \texttt{reconstruct\_graph\_at(t)} replay. What is missing is the \emph{measurement}: in the absence of a benchmark, neither a regulator nor a user can compare an audit-native runtime to a quickstart with default logging, nor compare two audit-native runtimes to each other. The agentic-systems survey of~\cite{wang2026evidencetrace} names exactly this gap.

We make four contributions:
\begin{enumerate}[leftmargin=*]
  \item \textbf{RAB v0.1.1 specification (§\ref{sec:spec}).} An abstract audit-log interface, three deterministic metrics, a scenario contract, a reporting format, five baselines, and six frozen honesty clauses (no LLM judge anywhere, log-only RF, operationalise-not-certify, re-verification artefacts, gap-as-headline, spec-versioning).
  \item \textbf{Two pre-registered scenarios (§\ref{sec:scenario}).} Scenario-S1 (\emph{lifecycle-small}, 40 ops, 10 checkpoints, 11 ingest / 4 update / 3 supersede / 2 delete / 20 query) is the v0.1 fixture. Scenario-S2 (\emph{lifecycle-large}, 400 ops, 40 checkpoints, 110 / 40 / 30 / 20 / 200, with nine supersede chains of average length 3.33 and longest length 5, and a cross-reference density of 3.90 doc-id mentions per content-bearing text body) was pre-registered before fixture construction and serves as the cross-scenario confirmation point. Both are deterministic and public-domain.
  \item \textbf{Adapter contract and four SUT implementations (§\ref{sec:adapters}).} Reference is a perfectly-audited in-memory SUT that pins the driver and scorer ($1.0\times 3$); JAMES exercises real production code paths (\texttt{emit\_lifecycle\_event} and \texttt{reconstruct\_graph\_at}), workspace-isolated from production state; Baseline-1 is a bolt-on OpenTelemetry-GenAI~\cite{otel2024genaisemconv} tracing baseline (the source spec for industry-standard LLM observability) with a hash-pinned mapping table published as a normative artefact; Baseline-0 is a vanilla in-memory RAG with Python-\texttt{logging}-style records (the default-logging floor).
  \item \textbf{Pre-registered measurement and gap table, replicated cross-scenario (§\ref{sec:results}).} Two reporting protocols, two cell-validity gate specifications, two sets of prohibitions (no post-hoc spec tweaks, no JAMES-wins framing, no Baseline-0 augmentation, no scenario-blending), and a locked honest-tier ladder were committed before any adapter code was written and before any number was produced. The pre-registration commits are the timestamped evidence. The 4-SUT measurement reveals two distinct fail-modes (§\ref{sec:results}) of non-audit-native instrumentation, neither of which the single-number AC overall column makes visible; every cell of the four-row gap table reproduces on scenario-S2 to the third decimal place, ruling out the "S1 might be small enough to cherry-pick" objection without expanding the framing post-hoc.
\end{enumerate}

The benchmark is the contribution. The audit-native architectural class is \emph{not} ours to claim --- ActiveGraph published the same architectural primitives independently three weeks before RAB R1.0~\cite{nakajima2026activegraph}; JAMES's relevant code (\texttt{core/lifecycle/replay\_audit.py}, \texttt{core/lifecycle/replay\_graph.py}) shipped in v0.4.2. Adjacent measurement axes also exist and are explicitly orthogonal to ours: DFAH~\cite{khatchadourian2026dfah} scores behavioural determinism (same input $\to$ same output reproducibility) at the agent-output level rather than at the log-artifact level, and COMPL-AI~\cite{guldimann2024complai} operationalises the EU AI Act at the model level (robustness, fairness, training-data traceability) rather than at the system-level record-keeping articles RAB targets. What we provide is the missing measurement instrument for the system-level record-keeping axis, and the first 4-SUT data point against it, including the bolt-on-tracing intermediate column that the AC-overall-only strawman objection would have been raised against.

\section{Related Work}
\label{sec:related}

\paragraph{Audit-native agent runtimes.} ActiveGraph~\cite{nakajima2026activegraph} explicitly identifies the append-only log as the source of truth, with the working graph as a deterministic projection. JAMES v0.4.2~\cite{seo2026jamesv043} ships \texttt{reconstruct\_graph\_at(t)} with the audit-only invariant pinned by a release-gating test. Both publications describe the same architectural class. RAB measures it.

\paragraph{Agentic-system trace surveys.} The survey of~\cite{wang2026evidencetrace} categorises trace-related work across reproducibility, auditability, and provenance, and names the open challenge of realistic execution-trace benchmarks. The benchmark category we occupy is the one the survey identifies as empty.

\paragraph{AI Act benchmarking and auditing.} Audit Cards~\cite{staufer2025auditcards} document \emph{how} an audit is contextualised; RAB measures \emph{what} the audit can reproduce. AIReg-Bench~\cite{airegbench2025} benchmarks LLM \emph{judges} of regulation compliance; RAB has no LLM judge anywhere in scoring (honesty clause~1). Casper et~al.~\cite{casper2024blackbox} argue that black-box access is insufficient for rigorous audits; RAB is a white-box instrument over the exported log and quantifies the gap that white-box access enables. The AI Act Evaluation Benchmark~\cite{aiactevalbench2026} evaluates classification triggers in the AI Act (manipulative AI, biometric ID, social scoring) at the system-categorisation level; RAB operates at the per-system audit-quality level. The two are orthogonal.

\paragraph{Provenance and risk-management frameworks.} The W3C PROV \texttt{wasDerivedFrom} relation~\cite{w3cprov2013} is the anchor for RAB's PC metric. The NIST AI~RMF~\cite{nist2023airmf} provides the broader governance vocabulary; RAB does not implement RMF function controls, but its scores are designed to feed an RMF \textsc{Measure} step.

\paragraph{RAG evaluation.} Most existing RAG benchmarks (cf.\ retrieval-augmented generation as introduced by~\cite{lewis2020rag} and the many domain-specific extensions) measure \emph{answer quality}. RAB measures \emph{audit quality} of the system that produced the answer. The two axes are complementary; a system can score well on RAG quality and fail RAB, or vice versa.

\paragraph{Behavioural determinism vs.\ log-artifact quality.} DFAH~\cite{khatchadourian2026dfah} measures whether a financial agent, replayed against the same input, produces the same output --- behavioural determinism, validated across 4{,}700+ runs over three financial benchmarks. The terminology overlap (``replayable'', ``audit'', ``harness'') deserves explicit disambiguation: DFAH and RAB are orthogonal axes, not competing instruments. DFAH scores \emph{what the agent outputs} given an input; RAB scores \emph{what the exported audit log lets you reconstruct} about that output's chain of evidence. Both axes can fail independently: a behaviourally deterministic agent can still emit an unauditable log (low RAB, high DFAH); an audit-native runtime can reproduce its log-only state and still drift in answer content between runs (high RAB, low DFAH). A complete record of a high-risk RAG or agent system arguably needs both numbers.

\paragraph{Model-level vs.\ system-level AI Act benchmarking.} COMPL-AI~\cite{guldimann2024complai} operationalises the EU AI Act into 12 technical requirements + 27 benchmarks at the \emph{model} level (robustness, fairness, traceability of training data). It is, to our knowledge, the first published benchmarking suite anchored to the AI Act; RAB inherits the operationalisation precedent. RAB targets the \emph{system-level} record-keeping obligations of Articles 10/12/19 that COMPL-AI does not score: COMPL-AI evaluates properties of the model artefact, while RAB evaluates the lifetime log artefact of the production system. The two are complementary tracks of a single Act, not substitutes --- a deployed system can fail RAB while running an AI-Act-compliant model, and vice versa.

\paragraph{Industry trace standards and LLM-as-debugger.} Industry-side tracing has produced two relevant artefact classes that bracket RAB rather than overlap it. OpenTelemetry's GenAI semantic conventions~\cite{otel2024genaisemconv} and adjacent platforms (LangSmith, MLflow) define \emph{format} standards for trace spans but do not score audit quality; they are the natural raw material for an RAB Baseline-1 SUT mapping bolt-on tracing onto Spec~§1 (this is the named comparison gap in §\ref{sec:future}). Industry compliance protocols that specify cryptographic audit-trail formats for AI Act compliance are likewise formats rather than scorers, and are natural candidate SUTs under RAB's adapter contract. LLM-as-debugger tools that use a model to read a trace and diagnose failure attribution measure the \emph{LLM's debugging capability}, not the trace's audit quality, and are explicitly compatible with RAB's honesty clause H1 (an LLM may consume an RAB-scoreable log without being part of the scorer).

\section{RAB v0.1.1 Specification}
\label{sec:spec}

The full spec is part of the release archive (\texttt{eval/rab/SPEC-v0.1.md}). We summarise its load-bearing pieces; the spec itself, not this paper, is the normative artefact.

\subsection{Abstract audit-log interface (Spec~§1)}
A system under test (SUT) is scoreable iff it can export its audit log as JSONL, one event per line, each line containing \texttt{event\_id}, ISO-8601 \texttt{ts}, \texttt{event\_type} (from the SUT's vocabulary; mapped once to RAB's canonical types via a declared mapping table that is part of the submission), \texttt{parent\_id} (provenance edge), \texttt{inputs\_hash}, and \texttt{payload}. RAB's canonical event vocabulary is \{\texttt{INGEST}, \texttt{UPDATE}, \texttt{SUPERSEDE}, \texttt{DELETE}, \texttt{RETRIEVE}, \texttt{RERANK}, \texttt{SYNTH}, \texttt{VERIFY}, \texttt{ANSWER}, \texttt{OTHER}\}. Unmapped decision-bearing events count against AC --- they were executed but not auditable in canonical terms.

\subsection{Metrics (Spec~§2)}

\paragraph{AC --- Audit Completeness (Art.~12(1)/(2)).} The RAB driver executes the published scenario against the SUT, recording ground truth $E_{\mathrm{exec}}$: every decision-bearing action requested. The SUT's exported log yields $E_{\mathrm{audit}}$. Matching is greedy, in time order: each executed op matches the first unconsumed log event of the same canonical type whose timestamp falls inside the op's $[t_{\mathrm{start}}, t_{\mathrm{end}}]$ window.
\[\mathrm{AC} = \frac{|E_{\mathrm{matched}}|}{|E_{\mathrm{exec}}|}\quad \text{reported overall and per canonical type.}\]

\paragraph{RF --- Replay Fidelity (Art.~12(2)(b)).} The scenario defines checkpoints $k = 1\ldots K$; at each checkpoint the driver snapshots ground-truth state $S_k$ via the SUT's live query interface. After the run, the SUT's \emph{replayer} is given only the exported log and must produce $R_k$.
\[
\mathrm{RF\text{-}exact} = \frac{|\{k : \mathrm{canon}(R_k) = \mathrm{canon}(S_k)\}|}{K}, \quad
\mathrm{RF\text{-}graded} = \frac{1}{K} \sum_k \mathrm{Jaccard}(\mathrm{items}(R_k), \mathrm{items}(S_k))
\]
A separate \textbf{RF-cost} column reports wall-clock seconds per~1000 log events folded during replay. It is reported alongside but never blended into the score; this is where the cost of audit-native scale enters the picture.

\paragraph{PC --- Provenance Coverage (Art.~10(2)(b), W3C PROV).} For every \texttt{ANSWER} event, its cited source identifiers must chain back through \texttt{parent\_id} links to an origin-bearing event (\texttt{INGEST} or \texttt{SUPERSEDE}, since both introduce content):
\[
\mathrm{ANSWER} \to \mathrm{SYNTH} \to \mathrm{RETRIEVE} \to (\mathrm{INGEST}\,|\,\mathrm{SUPERSEDE}).
\]
$\mathrm{PC} = \text{traceable citations} / \text{total citations}$. v0.1 scopes provenance to cited sources only; claim-level provenance is explicit future work.

\paragraph{Canonical form (Spec~§2.4).} State snapshots serialise as JSON with sorted keys, entity/edge arrays sorted by identifier, UTC ISO-8601 timestamps, floats fixed to 6~decimal places. The scorer's \texttt{canon()} function is identical for $S$ and $R$. RF-exact is byte-equality after canonicalisation; RF-graded uses the Jaccard on the item set (entity ids and \texttt{(src,dst,type)} edge triples).

\subsection{Honesty clauses (Spec~§6, frozen)}
\begin{enumerate}[label=H\arabic*,leftmargin=*]
  \item No LLM judge anywhere in scoring.
  \item RF consumes the exported log only --- no live-state access.
  \item RAB operationalises Art.~10/12/19 \emph{concepts}; it does not certify compliance, and the spec says so wherever scores are published.
  \item Scores published only with their re-verification artefacts (log, mapping, scorer version).
  \item The benchmark author's system scoring well is expected and is not the headline.
  \item Spec changes never retro-apply: results carry their spec version.
\end{enumerate}
Clause~H6 is what we used when correcting v0.1.0 to v0.1.1 during the reference-adapter implementation (Section~\ref{sec:adapters}).

\section{Scenarios}
\label{sec:scenario}

\subsection{Scenario-S1 (\emph{lifecycle-small})}
A 40-operation, 10-checkpoint lifecycle over a synthetic public-domain corpus about a fictional research lab (Northbridge Labs). Operations are 11 \texttt{INGEST}, 4 \texttt{UPDATE}, 3 \texttt{SUPERSEDE}, 2 \texttt{DELETE}, and 20 \texttt{QUERY}; identifiers are deterministic; the supersede chain produces both ``new-replaces-old'' and ``later-replaces-earlier-replaces-original'' shapes (longest chain length 2); deletes touch both ingested and ingested-then-updated documents.

\subsection{Scenario-S2 (\emph{lifecycle-large})}
A 400-operation, 40-checkpoint lifecycle over a synthetic public-domain corpus about a fictional city's operations (departments, projects, contracts, incidents, policies). Operations are 110 \texttt{INGEST}, 40 \texttt{UPDATE}, 30 \texttt{SUPERSEDE}, 20 \texttt{DELETE}, and 200 \texttt{QUERY}; identifiers follow the deterministic \texttt{co-XXX-NNN[-rN]} convention. The supersede plan locks in nine independent chains with an average length of $3.33$ and a longest chain of length~5; the cross-reference density (count of \emph{other} doc-id mentions per content-bearing text body) is~$3.90$, well above the pre-registration's $\ge 2.5$ sentinel. Both the chain shape and the cross-reference density were pinned in the pre-registration before the generator was written so that the larger graph would not be quietly tuned to the SUTs.

Both scenarios are deterministic and public-domain. Each is versioned and SHA-pinned; results cite \texttt{(spec~v0.1.1, scenario~S\{1,2\}, scenario-sha)}. We never blend scores across scenarios (locked prohibition; honesty clause~H6 forbids retro-application of spec changes, and the corresponding scenario-side prohibition forbids retro-fitting protocol claims by averaging across fixtures).

\section{Adapters}
\label{sec:adapters}

\subsection{Reference adapter}
A perfectly-audited in-memory SUT (Spec~§5). Its only purpose is to pin the driver and scorer: a system that logs every event with full payload and reconstructs state purely from its log \emph{must} score AC/RF-exact/PC $= 1.0$. Fault-injection variants (\texttt{drop\_audit\_types}, \texttt{corrupt\_replay}, \texttt{break\_provenance}) drop exactly the targeted metric, demonstrating that the three axes are independently observable. The reference adapter implementation surfaced a v0.1.0 defect in the PC origin-bearing rule (the original wording allowed \texttt{INGEST} only, marking supersede-born content untraceable). The defect was caught \emph{before} any measurement was taken; the spec changelog records the correction as v0.1.1. This is honesty clause~H6 in action.

\subsection{JAMES adapter}
The JAMES adapter (\texttt{eval/rab/adapters/james.py}) is workspace-isolated: every artefact (RAB log JSONL, lifecycle-event sqlite, optional vector store) lives under a constructor-supplied directory; production audit databases are never touched. Each \texttt{SUPERSEDE} additionally calls the real production code path \texttt{core.lifecycle.replay\_audit.emit\_lifecycle\_event} with \texttt{EVT\_SUPERSEDE\_EDGE\_CREATED}. Two cross-verification tests pin this bridge: the count of \texttt{SUPERSEDE} JSONL rows equals the count of \texttt{lifecycle.supersede.edge\_created} rows in the workspace sqlite; and \texttt{core.lifecycle.replay\_graph.reconstruct\_graph\_at(t)} reproduces the same supersede edges as the JSONL replay. The adapter therefore demonstrates that the published v0.4.3 JAMES code path actually scores what the gap table reports.

\subsection{Baseline-1 adapter (bolt-on OpenTelemetry GenAI tracing)}
Baseline-1 (\texttt{eval/rab/adapters/baseline1.py}) is the bolt-on-tracing intermediate point named in Spec~§5. The same in-memory RAG core that Baseline-0 runs is instrumented with OpenTelemetry GenAI semantic-conventions-shaped~\cite{otel2024genaisemconv} spans: each query emits a \texttt{retrieval} span (carrying \texttt{gen\_ai.retrieval.query} and \texttt{gen\_ai.retrieval.doc\_ids}) followed by a child \texttt{chat} span (carrying \texttt{gen\_ai.input.messages}, \texttt{gen\_ai.output.messages}, \texttt{gen\_ai.usage.input\_tokens}, \texttt{gen\_ai.response.citations}). The adapter does not depend on the OpenTelemetry SDK at runtime --- spans are dict-shaped per the source spec, which keeps the comparison cleanly a function of the mapping table, not of a particular OTel collector. The mapping table itself is a hash-pinned normative artefact, published at \texttt{eval/rab/mappings/otel\_genai\_to\_rab.json} and recorded as the run's \texttt{mapping\_table\_sha}. Its rationale paragraph in the table file documents (i) why \texttt{chat} maps to \texttt{ANSWER} rather than \texttt{SYNTH} under Spec~§1's 1:1 mapping requirement, and (ii) the structural absence of \texttt{INGEST}, \texttt{UPDATE}, \texttt{SUPERSEDE}, and \texttt{DELETE} in the OTel GenAI source spec --- a true gap in the source-spec coverage for AI-Act record-keeping purposes, and one of the headline findings of §\ref{sec:results}.

\subsection{Baseline-0 adapter}
Baseline-0 is the floor (Spec~§5). It is a vanilla in-memory RAG with Python-\texttt{logging}-style records: each operation emits a human string (e.g.~\texttt{"Indexed document nb-doc-001"}) with no structured payload, no \texttt{parent\_id} chain, and a mapping table that maps \texttt{doc\_added}~$\to$~\texttt{INGEST} but cannot distinguish updates, supersedes, deletes, or answers. Its \texttt{replay\_at} returns the empty state because the log carries no payload to fold. This is the honest representation of what most ``default logging'' looks like.

\section{Pre-registrations}
\label{sec:prereg}

Two measurements in this paper were pre-registered before adapter code or fixture content existed.

\paragraph{R1.4 (scenario-S1, 4-SUT gap).} The pre-registration document (\texttt{docs/research/r1-4-preregistration-2026-06-10.md}, repository commit \texttt{8a35dee}) locks the SUTs, the result-JSON shape (including \texttt{log\_sha}, \texttt{mapping\_table\_sha}, \texttt{scenario\_sha}), the cell-validity gates (reference must hit $1.0\times 3$ before publishing other SUTs' numbers; scenario errors recorded, not papered over), and six prohibitions: no post-hoc spec tweaks, no JAMES-wins framing, no Baseline-0 augmentation, no LLM judge anywhere, no live-state reads in RF, no auto-linking of the measurement to unrelated collaboration scopes. The honest tier ceiling for single-scenario evidence ($\star\star$) was declared before any number was produced.

\paragraph{R1.6 Phase~3 (scenario-S2, cross-scenario confirmation).} The Phase~3 pre-registration (\texttt{docs/research/r1-phase-3-scenario-s2-preregistration-2026-06-10.md}, repository commit \texttt{d21c680}, independently archived at Zenodo DOI~\href{https://doi.org/10.5281/zenodo.20635130}{10.5281/zenodo.20635130}) was committed \emph{before} the scenario-S2 fixture file existed and \emph{before} the \texttt{rab\_run.py} dispatch supporting S2 was written. It locks: (i)~the scenario shape (operation distribution exactly as quoted in §\ref{sec:scenario}, plus the chain and cross-reference sentinels); (ii)~the four SUTs (Reference, JAMES, Baseline-0, Baseline-1) with no addition or substitution; (iii)~an \emph{absolute prohibition on scenario blending} --- per-scenario numbers are reported, never averaged across S1 and S2; (iv)~a separate RF-cost hypothesis (cycle~$\gamma$'s ``graph build $O(N^2)$'' candidate, brought in from a sibling JAMES finding) with its own threshold pair: JAMES RF-cost~$\ge 0.05$~s/1k events \emph{and} S1$\to$S2 ratio~$\ge 5\times$ together promote the finding to $\star\star$, anything weaker leaves the column definition in the spec but withholds the claim; and (v)~a locked honest-tier ladder: $\star\star\star$ requires both the gap structure replicating \emph{and} the RF-cost finding clearing, $\star\star$ requires only gap replication, anything else is exploratory. The honest tier the paper now reports against the ladder is the second row ($\star\star$, gap replicated, RF-cost claim withheld) and is honest to the rule as written, not relaxed.

\section{Results}
\label{sec:results}

\begin{table}[t]
\centering
\caption{Gap table on RAB SPEC~v0.1.1 / scenario-S1. The four SUTs span the deployment spectrum from default logging to audit-native. Baseline-0 and Baseline-1 fail at different canonical categories: Baseline-0 catches AC \texttt{INGEST} but misses \texttt{ANSWER}; Baseline-1 catches \texttt{ANSWER} but misses \texttt{INGEST}. Both fail on RF and PC entirely. JAMES~=~Reference on S1 is \emph{expected} per honesty clause~H5; the headline is the per-canonical and per-axis structure of the gap, not the single AC-overall column.}
\label{tab:gap}
\small
\setlength{\tabcolsep}{4pt}
\begin{tabular}{lrrrrrrrrrr}
\toprule
\textbf{SUT} & \textbf{AC} & \textbf{INGEST} & \textbf{UPDATE} & \textbf{SUPERSEDE} & \textbf{DELETE} & \textbf{ANSWER} & \textbf{RF-exact} & \textbf{RF-graded} & \textbf{PC} & \textbf{events} \\
\midrule
Reference (gate)         & 1.000 & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 & 1.000 & 1.000 & 1.000 & 80 \\
JAMES (audit-native)     & \textbf{1.000} & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 & \textbf{1.000} & 1.000 & \textbf{1.000} & 80 \\
Baseline-1 (OTel bolt-on)& \textbf{0.500} & 0.00 & 0.00 & 0.00 & 0.00 & 1.00 & \textbf{0.000} & 0.000 & \textbf{0.000} & 40 \\
Baseline-0 (floor)       & \textbf{0.275} & 1.00 & 0.00 & 0.00 & 0.00 & 0.00 & \textbf{0.000} & 0.000 & \textbf{0.000} & 40 \\
\bottomrule
\end{tabular}
\end{table}

Table~\ref{tab:gap} is the v0.4.3 result. The single most important reading is the comparison between Baseline-0 and Baseline-1 in the per-canonical-type columns:

\paragraph{Two fail-modes, at different categories.} AC $=0.275$ (default logging) and AC $=0.500$ (bolt-on tracing) are not on the same axis. Both classes are deployed by operators today; both fail to meet the EU AI Act record-keeping obligations RAB scores; but they fail at \emph{different} canonical categories.

Baseline-0's failure mode: it catches the obvious thing (\texttt{logger.info("Indexed document …")} $\to$ AC \texttt{INGEST}~$=~1.0$), but its log carries strings, not events: it cannot distinguish ingest from update from supersede, does not record deletes structurally, and emits no canonical \texttt{ANSWER} event at all. AC \texttt{UPDATE}, \texttt{SUPERSEDE}, \texttt{DELETE}, \texttt{ANSWER}~=~0 are the consequences.

Baseline-1's failure mode is structurally opposite. The OpenTelemetry GenAI source spec targets runtime LLM-call observability: its \texttt{chat} span maps cleanly to \texttt{ANSWER} (AC \texttt{ANSWER}~$=~1.0$), but the source spec has no corpus-lifecycle vocabulary --- there is no \texttt{ingest\_document} or \texttt{supersede\_document} span in OTel GenAI. AC \texttt{INGEST}, \texttt{UPDATE}, \texttt{SUPERSEDE}, \texttt{DELETE}~=~0 follow directly from the source-spec coverage, not from any deficiency in the adapter or the mapping table. AC overall improves from 0.275 to 0.500, a real \(+0.225\), but the structural trade is \texttt{INGEST} observability for \texttt{ANSWER} observability.

\paragraph{49 citations emitted, 0 traceable.} Baseline-1's chat span carries \texttt{gen\_ai.response.citations}, and the mapping table surfaces them as the SPEC \texttt{payload.citations} field, so PC's denominator on this scenario is 49 (not zero). PC's numerator is zero: every \texttt{ANSWER}~$\to$~chain~$\to$~\texttt{RETRIEVE} chain walks correctly, but the chain has nowhere to terminate because no \texttt{INGEST} event exists in the log to be the origin. This is, in one number, why corpus-lifecycle vocabulary is load-bearing for provenance.

\paragraph{RF~=~0 across both non-audit-native tiers.} Both Baseline-0 (no payload at all) and Baseline-1 (OTel spans carry token counts and messages but not document state) replay to the empty state at every checkpoint. RF is the metric on which audit-native instrumentation is qualitatively separate from both non-audit-native classes: bolt-on tracing closes part of the AC gap but closes none of the RF gap.

\paragraph{JAMES~=~Reference on S1.} Expected per H5. Both top rows have AC/RF/PC~$=1.000$ across the board. The Reference adapter pins driver and scorer correctness; JAMES independently reproduces those numbers through real production code paths (\texttt{emit\_lifecycle\_event} and \texttt{reconstruct\_graph\_at}) under workspace isolation, with two cross-verification tests pinning the bridge.

\subsection{Scenario-S2 cross-scenario gap table}
\label{sec:results-s2}

\begin{table}[t]
\centering
\caption{Cross-scenario gap table: scenario-S2 (lifecycle-large, 400 ops, 800 log events for audit-native SUTs, 400 for non-audit-native). Every cell reproduces the S1 result of Table~\ref{tab:gap} to the third decimal place. The two fail-modes (Baseline-0 catches \texttt{INGEST} only; Baseline-1 catches \texttt{ANSWER} only; strictly disjoint canonical-type catches) are stable at $10\times$ scale. JAMES~=~Reference within scenario continues to hold; the scenario-system co-design caveat carried forward from §\ref{sec:limits} is therefore unchanged in shape but is now \emph{partially} mitigated by the cross-scale replication.}
\label{tab:gap-s2}
\small
\setlength{\tabcolsep}{4pt}
\begin{tabular}{lrrrrrrrrrr}
\toprule
\textbf{SUT} & \textbf{AC} & \textbf{INGEST} & \textbf{UPDATE} & \textbf{SUPERSEDE} & \textbf{DELETE} & \textbf{ANSWER} & \textbf{RF-exact} & \textbf{RF-graded} & \textbf{PC} & \textbf{events} \\
\midrule
Reference (gate)          & 1.000 & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 & 1.000 & 1.000 & 1.000 & 800 \\
JAMES (audit-native)      & \textbf{1.000} & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 & \textbf{1.000} & 1.000 & \textbf{1.000} & 800 \\
Baseline-1 (OTel bolt-on) & \textbf{0.500} & 0.00 & 0.00 & 0.00 & 0.00 & 1.00 & \textbf{0.000} & 0.000 & \textbf{0.000} & 400 \\
Baseline-0 (floor)        & \textbf{0.275} & 1.00 & 0.00 & 0.00 & 0.00 & 0.00 & \textbf{0.000} & 0.000 & \textbf{0.000} & 400 \\
\bottomrule
\end{tabular}
\end{table}

Table~\ref{tab:gap-s2} reports the scenario-S2 measurement. The numbers are identical, to the third decimal place, to the corresponding cells in Table~\ref{tab:gap}: same direction, same fail-modes, same per-canonical-type structure, with the scale of the log enlarged by an order of magnitude (40~$\to$~400 ops; 80~$\to$~800 events for audit-native SUTs, 40~$\to$~400 for non-audit-native). On this scenario, Baseline-1 emits $597$ citations across its $200$ answers, with the same $0$ traceable as on S1, because the same OTel-GenAI source-spec gap (no \texttt{INGEST} vocabulary) leaves the citation chain with no origin event to terminate at. This is not the kind of replication that survives mild perturbation by luck; it is what one expects when a gap is structural rather than scenario-specific.

\paragraph{What the cross-scenario replication does and does not show.} It rules out the simplest objection: that the v0.4.3 result was an artefact of scenario-S1's small size. It also strengthens the per-canonical disjoint fail-mode reading, since the same disjoint catches reappear on a fixture with a different domain (city operations rather than a research lab), a different chain shape (longest length~5 rather than~2), and a different cross-reference density. What it does \emph{not} do is mitigate the \emph{within-class} concern of §\ref{sec:limits}: JAMES~=~Reference still holds on S2, but the canonical vocabulary that S2 uses is the same one S1 uses, and the same one JAMES's internal lifecycle taxonomy uses; ruling out unconscious shaping of the spec by the SUT it most closely matches needs a future scenario whose event-vocabulary basis differs (queued as scenario-S3 cross-lingual in §\ref{sec:future}) and the replication-invite track to runtimes whose taxonomies were defined independently.

\paragraph{RF-cost (the scale-axis hypothesis).} The pre-registration's §4 hypothesised that JAMES's RF-cost (wall-clock seconds per~1000 log events folded during replay) would activate visibly at the larger scale, with two thresholds locked: $\ge 0.05$~s/1k events \emph{and} an S1$\to$S2 ratio of $\ge 5\times$ together promote the finding to $\star\star$; either condition unmet leaves the column definition in the spec but withholds the claim. The observed JAMES RF-cost is $0.0387$~s/1k events on S1 and $0.1081$~s/1k events on S2 --- the first sentinel clears (current cost is above the $0.05$ threshold) but the ratio is $2.79\times$, well below the locked $5\times$. The Reference adapter's RF-cost grows from $0.0013$ to $0.0051$~s/1k events, a $3.92\times$ ratio in the same neighbourhood. The honest reading is that a single S2 data point cannot distinguish "linear-ish with a constant factor" from "an early stretch of an $O(N^2)$ curve that has not yet compounded"; the locked decision applies and the claim is withheld. RF-cost remains a defined column of the spec and the present numbers are honest data points, but the cycle-$\gamma$ candidate finding does not promote on the basis of two scenarios in the present ratio range. A future scenario at $\ge 4\times$ the S2 operation count would be the cleanest next data point.

\textbf{Honest tier:} $\star\star$ cross-scenario partial: gap structure confirmed across S1 and S2 to the third decimal place; RF-cost finding not confirmed; cross-vocabulary-basis and external-replication mitigations of the scenario-system co-design caveat queued in §\ref{sec:future}. $\star\star\star$ is the ladder's top rung; the pre-registration locked it to require \emph{both} the gap and the RF-cost leg, and this paper reports against the ladder as written.

\section{Limitations and Threats to Validity}
\label{sec:limits}

We declare the limitations of v0.1.1 deliberately and in advance of release, per honesty clause~H5:
\begin{itemize}[leftmargin=*]
  \item \textbf{Two same-vocabulary scenarios; cross-bench still future.} v0.1.1 now ships two scenarios at different scales (S1 at 40 ops, S2 at 400 ops). The cross-scenario replication on S2 rules out the simplest objection (S1 might have been small enough to cherry-pick) but does \emph{not} address vocabulary-basis variation: both fixtures use the same canonical event set \{INGEST, UPDATE, SUPERSEDE, DELETE, ANSWER\}. Generalisation to multilingual content (scenario-S3 pre-registered separately at the §\ref{sec:future} track), to long-running production traces, and to adversarial inputs is future work. Adding a scenario whose vocabulary basis is independently chosen is the path to $\star\star\star$. In particular, both S1 and S2's \texttt{e\_exec} ground-truth canonical sets exclude RETRIEVE / SYNTH / RERANK: AC does not score them, even though SUTs may emit them. Baseline-1 does emit \texttt{RETRIEVE} events in its log (49 on S1, 597 on S2), but those events score zero towards AC because they do not match a ground-truth op; a future scenario that lifts retrieval to driver-side ground truth would let bolt-on tracing earn credit on that axis.
  \item \textbf{Three SUT classes, no third-party audit-native runtime.} The gap is now audit-native vs.\ bolt-on tracing vs.\ default-logging, but the audit-native column has only one occupant (JAMES) measured by the benchmark authors. A second audit-native point from a runtime whose taxonomy was defined independently --- ActiveGraph~\cite{nakajima2026activegraph} is the obvious candidate --- is the strongest available mitigation of the within-class self-evaluation concern and is the highest-priority replication invite (§\ref{sec:future}).
  \item \textbf{Cited-source provenance, not claim-level.} PC v0.1 traces source identifiers, not the claim a citation supports. Claim-level provenance is a future spec version with its own honesty review.
  \item \textbf{Synthetic prose.} Northbridge Labs prose is public-domain and licence-friendly but does not exercise PII redaction, multilingual disambiguation, or table-and-figure citation. The benchmark is designed to be extended, not assumed to cover these axes today.
  \item \textbf{Author-of-benchmark SUT.} JAMES is one of four SUTs we measure. Honesty clauses~H3 and H5, the gap-as-headline reporting protocol, and the inclusion of Baseline-1 (which directly tests whether the AC-overall column tells the whole story) together control for this conflict of interest. A peer-replication track is anticipated (R1.6, separate scope).
  \item \textbf{Scenario-system co-design risk --- partially mitigated by cross-scenario, vocabulary-basis still open.} The canonical event vocabulary (\texttt{INGEST}, \texttt{UPDATE}, \texttt{SUPERSEDE}, \texttt{DELETE}) shared by S1 and S2 is isomorphic to JAMES's own lifecycle taxonomy. JAMES's $1.000\times3$ on both scenarios is therefore part-by-design and part-by-measurement: the gap to Baseline-0 / Baseline-1 is informative regardless, and the cross-scenario replication on S2 (Table~\ref{tab:gap-s2}) rules out the simplest size-cherry-picking objection. The within-class equality (JAMES~=~Reference within scenario) still cannot, on two same-vocabulary scenarios, rule out unconscious shaping of the spec by the SUT it most closely matches. Two independent mitigations remain queued as Future Work (§\ref{sec:future}): scenario-S3 with a different event-vocabulary basis (cross-lingual), and external replication invites to runtimes whose taxonomies were defined independently of JAMES's. The Baseline-1 mitigation listed in the v1.0 draft of this paper has since been delivered; the scenario-S2 cross-scenario mitigation listed in v1.0--v1.2 has likewise been delivered (this revision). The remaining within-class concern is the audit-native column on a same-vocabulary scenario family, and the queued vocabulary-basis variant is the targeted mitigation.
  \item \textbf{Mutation-site wiring is partial in v0.4.3.} The JAMES SUT's $1.000$ holds for the workspace-isolated RAB adapter path; v0.4.2's published audit-only invariant requires a follow-up cross-cutting cycle to wire every production mutation site (\texttt{T1}, \texttt{T2}, \texttt{T2.D}, \texttt{T6}, \texttt{T7}) through \texttt{emit\_lifecycle\_event}. Until that cycle lands, RAB's JAMES score reflects what the RAB adapter exercises rather than every production code path. The wiring follow-up is queued; once it lands, an additional measurement (\texttt{james-production-S1}) can be reported alongside the workspace-adapter number.
\end{itemize}

\section{Reproducibility}
\label{sec:repro}

All artefacts are released under Zenodo concept DOI \href{https://doi.org/10.5281/zenodo.20625533}{10.5281/zenodo.20625533} (v0.4.3 version-specific DOI; the scenario-S2 measurement is post-v0.4.3, on commit \texttt{753022b}, scheduled to be archived under a forthcoming v0.4.4 DOI). The scenario-S2 pre-registration is independently archived at Zenodo DOI~\href{https://doi.org/10.5281/zenodo.20635130}{10.5281/zenodo.20635130}. The release archive plus the scenario-S2 measurement commit contain:
\begin{itemize}[leftmargin=*]
  \item \texttt{eval/rab/SPEC-v0.1.md} --- the frozen specification.
  \item \texttt{eval/rab/scenarios/\{s1\_lifecycle\_small.json,s2\_lifecycle\_large.json\}} --- both scenarios, hash-pinned. The S2 fixture is reproducible from the bundled generator (\texttt{scripts/research/build\_rab\_scenario\_s2.py}); the generator's announced byte-string SHA matches the fixture content, and the \texttt{scenario\_sha} recorded in every S2 \texttt{result.json} is the SHA returned by \texttt{Path.read\_text(encoding='utf-8')} (universal-newline normalised) so that all SUT runs agree on the same number on every platform.
  \item \texttt{eval/rab/\{driver,scorer\}.py}, \texttt{eval/rab/adapters/\{reference,baseline0,baseline1,james\}.py} --- the deterministic driver, scorer, and four adapters.
  \item \texttt{eval/rab/mappings/otel\_genai\_to\_rab.json} --- the hash-pinned mapping table for the OpenTelemetry-GenAI Baseline-1 SUT, recorded as \texttt{mapping\_table\_sha} in every Baseline-1 \texttt{result.json}.
  \item \texttt{reports/rab/<sut>-S\{1,2\}-<ts>.\{result.json,log.jsonl,mapping.json\}} --- the SPEC~§4 re-verification triple for each SUT on each scenario (eight triples total: four SUTs $\times$ two scenarios). Re-running the scorer on the same artefacts reproduces the numbers bit-for-bit.
  \item \texttt{tests/test\_rab\_\{benchmark,baseline0,baseline1,james\_adapter,scenario\_s2\}.py} --- 51 tests pinning self-verification, the floor, the bolt-on tier, the audit-native bridge, and the scenario-S2 fixture shape (op distribution / chain shape / cross-reference density / Reference $1.000\times3$ gate).
\end{itemize}
The minimal reproduction recipe is:
\begin{lstlisting}
git clone https://github.com/Hashevolution/James-RAG-Evol
cd James-RAG-Evol && git checkout 753022b   # v0.4.3 + scenario-S2
python -m pytest tests/test_rab_benchmark.py \
                 tests/test_rab_baseline0.py \
                 tests/test_rab_baseline1.py \
                 tests/test_rab_james_adapter.py \
                 tests/test_rab_scenario_s2.py
for sut in reference james baseline1 baseline0; do
  python scripts/research/rab_run.py --sut $sut --scenario S1
  python scripts/research/rab_run.py --sut $sut --scenario S2
done
\end{lstlisting}

\section{Future Work}
\label{sec:future}

Four explicit tracks remain out of scope for v0.1.1 after the Baseline-1 SUT (v1.2 revision) and the scenario-S2 cross-scenario confirmation (this revision) were delivered:
\begin{enumerate}[leftmargin=*]
  \item \textbf{Scenario-S3 (cross-lingual) and beyond.} A multilingual corpus (so the canonical-event vocabulary is no longer single-language, addressing the targeted residue of the scenario-system co-design concern in §\ref{sec:limits}), an adversarial-input track, and a scenario at $\ge 4\times$ the scenario-S2 operation count to provide a clean third data point on the RF-cost axis (where the $\star\star$ promotion threshold remains unmet at the current scenario range). A future scenario that lifts RETRIEVE / SYNTH / RERANK to driver-side ground truth would additionally let Baseline-1's emitted-but-currently-unscored RETRIEVE events contribute to AC, providing a finer measurement of where on the per-canonical axis bolt-on tracing earns credit.
  \item \textbf{Replication invites.} We will invite the authors of ActiveGraph~\cite{nakajima2026activegraph} and DFAH~\cite{khatchadourian2026dfah}, and other audit-native runtimes and trace-scoring teams, to submit SUT adapters and contribute results. This is a separate collaboration scope; results from such replications, when they arrive, will be reported under the same Spec~§4 protocol. A second audit-native point from a runtime whose taxonomy was defined independently of JAMES's is the single strongest available mitigation of the within-class self-evaluation concern that remains after the cross-scenario replication.
  \item \textbf{Mutation-site wiring follow-up.} Lifting JAMES's $1.000$ from the RAB adapter path to every production mutation site (the v0.4.2 carry-over described in §\ref{sec:limits}) and reporting a separate \texttt{james-production-S\{1,2\}} number alongside the workspace-adapter ones.
  \item \textbf{SPEC v0.1.2 candidate items.} A normative clarification of \texttt{scenario\_sha}'s computation (i.e.\ explicitly anchored to the universal-newline-normalised \texttt{Path.read\_text} form so that cross-platform agreement is part of the spec rather than an adapter-level convention) is the cleanest candidate single change. A scenario-size-guidance annotation in §3 is a possible accompanying clarification.
\end{enumerate}

\section{Conclusion}

RAB v0.1.1 is, to our knowledge, the first benchmark that scores the exported audit-log artifact of arbitrary RAG and agent systems against the system-level record-keeping articles of the EU AI Act --- the axis left open by behavioural-determinism harnesses~\cite{khatchadourian2026dfah} and model-level AI Act benchmarking suites~\cite{guldimann2024complai}. It does not introduce a new architecture; ActiveGraph~\cite{nakajima2026activegraph} and JAMES~\cite{seo2026jamesv043} already realise the architectural class, independently. What it introduces is a deterministic, pre-registered, externally-anchored benchmark with a frozen specification and a published re-verification protocol, just before the EU AI Act's high-risk obligations enter force. The four-SUT measurement on two pre-registered scenarios at an order-of-magnitude scale gap makes the result's shape, not its single-column AC overall, the headline: default logging and bolt-on tracing both fail to meet the AI Act record-keeping bar, but they fail at \emph{different} canonical categories; both fail completely on the RF and PC axes that audit-native instrumentation occupies alone; and the four-row gap structure reproduces from scenario-S1 to scenario-S2 to the third decimal place, ruling out the simplest size-cherry-picking objection without expanding the framing post-hoc. The honest-tier ladder commits the headline to the second rung ($\star\star$ cross-scenario partial), because the separately pre-registered RF-cost hypothesis is below threshold and the within-class self-evaluation concern is only partially mitigated. The gap table is the headline. The benchmark is the contribution.

\section*{Author contributions and conflicts of interest}
Single-author work by Jiwon Seo (Hashevolution). The author is also the maintainer of PROJECT JAMES, one of the SUTs measured. Honesty clauses~H3 and H5, the pre-registered reporting protocol, the workspace-isolated JAMES adapter (production audit databases not touched), and the inclusion of a self-verifying reference SUT together control for this conflict of interest. The same benchmark applied to other audit-native runtimes is explicitly invited (Section~\ref{sec:future}).

\section*{Data and code availability}
All code, the specification, the scenario fixture, the deterministic scorer, the three adapters, the pre-registration document, the cross-verification tests, and the 9 measurement artefacts are publicly available under the MIT license at the JAMES repository, release tag \texttt{v0.4.3}, and mirrored on Zenodo at \href{https://doi.org/10.5281/zenodo.20625533}{10.5281/zenodo.20625533}.

\section*{Acknowledgements}
This work was developed in dialogue with Claude (Anthropic) as a pair-collaborator. The author thanks LEO~(Younghu, AIS Edu HK) for the foundational research collaboration that shaped the JAMES platform on which RAB's audit-native SUT is built.

\bibliographystyle{alpha}
\bibliography{refs}

\end{document}
