% LRB v0.2 — arXiv preprint draft
% Build:  pdflatex main && bibtex main && pdflatex main && pdflatex main
% Mirrors papers/rab-preprint/main.tex structure.
\documentclass[11pt,a4paper]{article}
\usepackage[margin=1in]{geometry}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{microtype}
\usepackage{booktabs}
\usepackage{array}
\usepackage{enumitem}
\usepackage{hyperref}
\usepackage{url}
\usepackage{xcolor}
\usepackage{listings}
\usepackage{amsmath}
\usepackage{caption}
\hypersetup{
  colorlinks=true,
  linkcolor=blue!50!black,
  citecolor=blue!50!black,
  urlcolor=blue!50!black,
  pdftitle={LRB: A Lifecycle Retrieval Benchmark for Temporal RAG},
  pdfauthor={Jiwon Seo},
  pdfkeywords={RAG, lifecycle, temporal retrieval, validity window, audit log, benchmark}
}
\lstset{
  basicstyle=\ttfamily\small,
  breaklines=true,
  columns=fullflexible,
  frame=single,
  framerule=0.3pt,
  xleftmargin=4pt,
  xrightmargin=4pt
}

\title{LRB: A Lifecycle Retrieval Benchmark for Temporal RAG\thanks{Pre-registrations locked before any measurement (file path + first-commit SHA in the JAMES repository, all archived at Zenodo DOI \href{https://doi.org/10.5281/zenodo.20652679}{10.5281/zenodo.20652679} v0.4.4): Phase A protocol at \texttt{docs/research/lrb-phase-a-smoke-preregistration-2026-06-11.md} (commit \texttt{3c32743}); Phase B (time-travel axis + 3-SUT) at \texttt{docs/research/lrb-phase-b-time-travel-preregistration-2026-06-11.md} (commit \texttt{c0c5956}); v0.2.1 cross-model at \texttt{docs/research/lrb-v021-cross-model-preregistration-2026-06-11.md} (commit \texttt{0e60960}); v0.2.4 HR axis at \texttt{docs/research/v024-hr-nli-axis-preregistration-2026-06-11.md} (commit \texttt{8f2be4f}); v0.2.3 S3 publication-scale at \texttt{docs/research/lrb-v023-s3-publication-scale-preregistration-2026-06-12.md} (commit \texttt{3896ac1}); v0.2.3b S3 cross-model at \texttt{docs/research/lrb-v023b-s3-cross-model-preregistration-2026-06-12.md} (commit \texttt{a2a471e}).}}

\author{
  Jiwon Seo\thanks{Hashevolution, Republic of Korea. ORCID 0009-0002-0007-7860. Correspondence: \texttt{karu-7@hanmail.net}.}
}

\date{2026-06-12}

\begin{document}

\maketitle

\begin{abstract}
Real-world enterprise corpora change over time: policies are amended, directors are replaced, contracts are renegotiated. Standard retrieval-augmented generation (RAG) benchmarks (MuSiQue, 2WikiMultiHop, ALCE, TimeQA) evaluate on \emph{frozen} corpora and therefore cannot measure whether a system retrieves the \emph{time-valid} version of a document. We present \textbf{LRB} (Lifecycle Retrieval Benchmark) v0.2, a pre-registered deterministic benchmark for the \emph{temporal validity} axis of RAG. LRB ships three scenarios (S1 lifecycle-quarterly, S2 lifecycle-yearly-with-time-travel, S3 publication-scale with programmatic vocabulary) with 100/200/1000 documents undergoing 160/564/5620 INGEST/UPDATE/SUPERSEDE/DELETE events across 12/24/52 weeks. Each query carries a \texttt{(query\_time, valid\_time)} pair so the benchmark can score \emph{current-state} retrieval (\texttt{query\_time = valid\_time}) and \emph{time-travel} retrieval (\texttt{query\_time > valid\_time}) on the same fixture. We score seven deterministic axes (R@5, R@10, P@5, P@10, latency, token cost, temporal\_accuracy) and three exploratory top-1 axes; an optional Hallucination Resistance (HR) axis is computed via NLI entailment with two checkpoint-pinned verifiers. We measure three SUTs: a vanilla append-only RAG (Baseline-0), a naive supersede-aware RAG that removes superseded documents (Baseline-1), and JAMES, an audit-native runtime with per-document validity windows. \textbf{Headline gap (S2, R@1)}: token-mode 0.225/0.5375/0.7125 across V/N/J; under LLM-grounded rerank, the V $<$ N $<$ J rank order is preserved across four models (gemma4:e4b 4B, gemma3:12b 12B, mixtral:8x7b 47B, claude-haiku-4-5). \textbf{Cross-scale (S2 $\to$ S3, R@1)}: V $<$ N $<$ J is preserved at every cell of a 4-point scale ladder spanning a 12.5$\times$ jump (S2 N=80 $\to$ S3 publication N=1000) with the JAMES $-$ Naive gap above $+$0.10 throughout (S3 publication: V/N/J $=$ 0.502/0.721/0.845). The benchmark, the six pre-registrations, and all 80+ measurement artefacts are committed in the JAMES repository; the headline is the \emph{gap structure}, not any single system's score. LRB does not claim the validity-window architecture is novel (cf.~\cite{nakajima2026activegraph}); the benchmark, the lifecycle-versus-current decomposition, and the three-scenario / four-model / four-scale / two-NLI cross-validation are the contribution.
\end{abstract}

\section{Introduction}
\label{sec:intro}

A growing fraction of enterprise RAG deployments operates over corpora that change continuously: legal contracts get amended, organisational directors get replaced, compliance policies get superseded, vendor agreements get renegotiated. The de-facto evaluation pipeline for RAG --- frozen-fixture multi-hop QA --- cannot measure whether a system retrieves the \emph{time-valid} document. A vanilla append-only retriever may surface both old and new versions of a superseded document; the user sees a confused mixture. A naive supersede-aware retriever that removes the old version on update is correct on \emph{current-state} queries but loses the ability to answer historical ones (``what was the policy when the contract was signed?''). An audit-native runtime with per-document validity windows can answer both. Today there is no public benchmark to discriminate these three regimes.

LRB fills this gap. The benchmark is the \emph{measurement infrastructure}, not the system; LRB does not claim that any one of the three SUT regimes is novel (an event-sourced-log architecture with deterministic replay was independently demonstrated by \cite{nakajima2026activegraph}; lifecycle-aware retrieval is well-established in industry data warehouses). The contribution is:

\begin{enumerate}[leftmargin=*]
  \item \textbf{Two pre-registered scenarios} (S1 lifecycle-quarterly, S2 lifecycle-yearly-with-time-travel), with deterministic generators reproducible from the JAMES repository.
  \item \textbf{An explicit \texttt{(query\_time, valid\_time)} query interface} that separates current-state retrieval from time-travel retrieval on the same fixture.
  \item \textbf{Seven deterministic scoring axes} plus three exploratory top-1 axes, with no LLM judge anywhere in the scoring pipeline.
  \item \textbf{Cross-model validation across four model families} (gemma4:e4b, gemma3:12b, mixtral:8x7b, claude-haiku-4-5) under both token-overlap-only and LLM-grounded retrieval modes.
  \item \textbf{An optional HR (Hallucination Resistance) axis} with two NLI verifiers (RoBERTa-large-MNLI primary, DeBERTa-v3-large-mnli-fever-anli-ling-wanli secondary) for cross-verifier agreement.
  \item \textbf{Cross-bench reproducibility check} against MuSiQue (Trivedi et al.~2022) to verify the SUT differentiation \emph{is} task-specific (LRB lifecycle queries) vs.~\emph{is not} task-specific (closed-book multi-hop).
\end{enumerate}

LRB is positioned as a sibling axis to RAB~\cite{seo2026rab}, which measures the \emph{audit log} the system exports rather than the \emph{retrieval quality} the system delivers. The two benchmarks share a discipline (pre-registration, deterministic scoring, gap-table headline) and a research framing (operationalising EU AI Act-style record-keeping requirements in measurable benchmarks).

\section{Related Work}
\label{sec:related}

\paragraph{Multi-hop and closed-book QA benchmarks.} MuSiQue~\cite{trivedi2022musique} and 2WikiMultiHop~\cite{ho2020wikimultihop} measure multi-step reasoning on frozen Wikipedia paragraphs. ALCE~\cite{gao2023alce} measures citation faithfulness on ASQA-style answers. These benchmarks do not vary the corpus across queries; the validity-window axis is not on their evaluation grid. We re-run MuSiQue dev (n=20) on the three LRB SUTs as a \emph{cross-bench reproducibility check} in Section~\ref{sec:cross_bench}: all three SUTs produce bit-for-bit identical EM/F1/support\_recall scores at gemma3:12b, confirming that LRB's SUT differentiation is task-specific (closed-book questions don't activate the lifecycle mechanism).

\paragraph{Temporal QA benchmarks.} TimeQA~\cite{chen2021timeqa}, TempReason~\cite{tan2023tempreason}, and TimeBench~\cite{chu2024timebench} measure temporal reasoning on Wikidata-derived time-stamped facts. These benchmarks focus on \emph{when the question is asked} (the LLM's temporal reasoning over given facts), not \emph{when the corpus is valid} (the retrieval system's lifecycle awareness). LRB's \texttt{(query\_time, valid\_time)} pair is closer to the bitemporal axis used in temporal databases~\cite{snodgrass1995tsql} than to the timestamp-stamped question axis of TimeQA.

\paragraph{Audit and replay benchmarks.} RAB~\cite{seo2026rab} measures audit-log quality (AC/RF/PC). DFAH~\cite{khatchadourian2026dfah} measures behavioural determinism of agentic systems. LLM-as-investigator and LLM-as-debugger observability tools measure trace-reading capability rather than the trace's intrinsic audit quality. These all measure the artefacts a system \emph{exports} after running (or downstream LLM analyses of those artefacts), not the retrieval quality \emph{during} the run. LRB's HR axis is the closest overlap, measuring whether generated answers are entailed by retrieved context, but the primary LRB axes are retrieval-side.

\paragraph{Lifecycle-aware runtimes.} ActiveGraph~\cite{nakajima2026activegraph} demonstrates an event-sourced log with deterministic replay. JAMES~\cite{jamesrepo} ships a similar architecture (independently developed, per RAB paper~\cite{seo2026rab} \S2). LRB is benchmark-neutral and treats both ActiveGraph and JAMES as candidate audit-native SUTs; the present paper measures only the JAMES adapter for the audit-native SUT row (ActiveGraph requires a separate adapter; we list this as future work \S\ref{sec:future}).

\section{Benchmark Specification}
\label{sec:spec}

% TODO: full spec — scenario fixture shape, event types, query format,
% scoring axes, honest tier ladder.
% Reference: docs/design/v0.4-lrb-lifecycle-retrieval-benchmark-design.md
% + all four pre-reg docs.

\paragraph{Scenarios.} LRB v0.2 ships two scenarios:

\begin{itemize}[leftmargin=*]
  \item \textbf{S1 (lifecycle-quarterly)}: 100 initial documents, 12 weeks, 160 lifecycle events, 60 queries $\times$ 3 timestamps (T=0, T=6w, T=12w) = 180 evaluations. Designed for \emph{current-state} retrieval evaluation.
  \item \textbf{S2 (lifecycle-yearly-with-time-travel)}: 200 initial documents, 24 weeks, 564 lifecycle events, 80 queries with explicit \texttt{(query\_time, valid\_time)} pairs covering four query types (current 40, historical-mid 20, historical-early 10, never-stale 10).
\end{itemize}

Both fixtures derive their vocabulary from the RAB scenario-S2 city-operations corpus (license friction 0), with sha-pinned JSON outputs reproducible from \texttt{scripts/research/build\_lrb\_scenario\_s\{1,2\}.py}.

\paragraph{Lifecycle event types.} INGEST (add document), UPDATE (in-place revision), SUPERSEDE (replace document at a specific week; the old version remains valid up to week-1 of the supersede, the new version becomes valid from the supersede week), DELETE (mark document invalid from a week). The SUTs differ in how they handle SUPERSEDE: Vanilla keeps both versions; Naive removes the old version on supersede; JAMES annotates per-document validity windows and applies the time-travel filter at query time.

\paragraph{Scoring axes (seven deterministic).} R@5, R@10, P@5, P@10, retrieval latency, retrieved-context token cost (proxy: characters $/$ 4), temporal\_accuracy (strict: R@k $= 1.0$ for the query's gold-at-valid\_time set). All axes are pure-functional on the per-query rows; no LLM judge.

\paragraph{Exploratory top-1 axes (three).} R@1, P@1, temporal\_accuracy\_strict\_top1. Critical for evaluating the user-visible top result.

\paragraph{HR axis (optional, v0.2.4).} Atomic claim extraction from the LLM-generated answer (rule-based plus optional LLM augmentation), per-claim NLI entailment against the retrieved-context premise, fraction of claims labelled \emph{entailment} as HR score. Two checkpoint-pinned verifiers (RoBERTa-large-MNLI primary, DeBERTa-v3-large-mnli-fever-anli-ling-wanli secondary) for cross-verifier agreement. The HR axis is the only LRB axis that touches an LLM, and only at the scoring step (not the retrieval step), preserving RAB-H1 (no LLM judge in scoring pipeline) at the level of the retrieval and SUT contract.

\section{SUTs}
\label{sec:suts}

The three SUTs are deliberately minimal: they share the same TF-IDF token-overlap base retriever (deterministic; tie-broken by \texttt{doc\_id}) and differ \emph{only} in how they handle the SUPERSEDE event.

\paragraph{Vanilla (append-only).} \texttt{ingest}, \texttt{update}, \texttt{supersede}, \texttt{delete} all map onto a flat document index. On SUPERSEDE both the old version and the new version remain in the index (the ``classic quickstart'' behaviour). DELETE removes the document. The retriever does not honour \texttt{valid\_time}.

\paragraph{Naive-supersede.} Identical to Vanilla except that on SUPERSEDE the old document is removed from the index and the new document inserted. The retriever does not honour \texttt{valid\_time}; ``time-travel queries'' from a later vantage point cannot recover the prior state.

\paragraph{JAMES (validity-window).} Each document carries an explicit \texttt{(valid\_from, valid\_to)} window. SUPERSEDE caps the old document's window at \texttt{week-1} and inserts the new document with \texttt{valid\_from} $=$ \texttt{week}. DELETE caps the window at \texttt{week-1}. At retrieval time, only documents valid at the query's \texttt{valid\_time} enter the candidate pool. JAMES is the only SUT that can answer time-travel queries.

All three SUTs expose the same \texttt{retrieve\_at(q, k, query\_time, valid\_time)} interface; the Vanilla and Naive implementations ignore \texttt{valid\_time}. A \texttt{get\_doc(doc\_id)} read-only accessor on all three feeds the optional cross-model LLM rerank wrapper (v0.2.1).

\section{Experiments}
\label{sec:experiments}

We report measurements committed under \texttt{reports/external/lrb/} of the JAMES repository (Apache-2.0 license; each cell ships \texttt{result.json} $+$ \texttt{bench.jsonl} with sha-pinned fixture hashes). All scores below are the deterministic output of the runner; re-running on the same fixture sha reproduces them bit-for-bit.

\subsection{Phase A: current-state retrieval (S1)}
\label{sec:phase_a}

LRB-S1 has 60 queries $\times$ 3 timestamps $=$ 180 evaluations on a 100-document corpus with 12 weeks of evolution. All queries are current-state (\texttt{query\_time} $=$ \texttt{valid\_time}).

\begin{table}[h]
\centering
\caption{LRB-S1 token-mode results (no LLM in the loop). Naive-supersede $\equiv$ JAMES on every axis: on current-state queries, JAMES's validity-window mechanism is task-orthogonal to Naive's remove-on-supersede mechanism --- both surface the same per-T live document set.}
\label{tab:phase_a}
\begin{tabular}{l c c c c}
\toprule
\textbf{SUT} & \textbf{R@5} & \textbf{R@10} & \textbf{temporal\_accuracy} & \textbf{R@1 (exp)} \\
\midrule
Vanilla & 0.844 & 0.894 & 0.894 & 0.617 \\
Naive-supersede & 0.844 & 0.917 & 0.917 & 0.739 \\
JAMES & 0.844 & 0.917 & 0.917 & 0.739 \\
\bottomrule
\end{tabular}
\end{table}

\textbf{Phase A honest finding.} On current-state queries (\texttt{query\_time} $=$ \texttt{valid\_time}), Naive-supersede achieves the same retrieval as JAMES at every axis. The validity-window's marginal contribution on this scenario is \emph{zero}. This motivated Phase B.

\subsection{Phase B: time-travel retrieval (S2)}
\label{sec:phase_b}

LRB-S2 has 80 queries on a 200-document corpus with 24 weeks of evolution; queries cover four types (current 40, historical-mid 20, historical-early 10, never-stale 10) with explicit \texttt{(query\_time, valid\_time)} pairs.

\begin{table}[h]
\centering
\caption{LRB-S2 token-mode results. Vanilla can retrieve historical documents (it never deleted them), but ranks them indistinguishably from current versions $\rightarrow$ R@1 floor 0.225. Naive cannot reach prior-T states (deleted on supersede) $\rightarrow$ R@10 floor 0.75 on historical types. JAMES uniquely both \emph{retains} prior versions \emph{and} \emph{discriminates} by valid\_time.}
\label{tab:phase_b}
\begin{tabular}{l c c c c}
\toprule
\textbf{SUT} & \textbf{R@5} & \textbf{R@10} & \textbf{temporal\_accuracy} & \textbf{R@1 (exp)} \\
\midrule
Vanilla & 0.875 & 0.950 & 0.950 & 0.225 \\
Naive-supersede & 0.663 & 0.750 & 0.750 & 0.538 \\
\textbf{JAMES} & \textbf{0.900} & \textbf{0.975} & \textbf{0.975} & \textbf{0.713} \\
\bottomrule
\end{tabular}
\end{table}

The rank \textbf{V $<$ N $<$ J on R@1} is the publishable headline for the time-travel axis. Per-category breakdown (Table~\ref{tab:phase_b_cat}) localises the gap: on \texttt{historical-early-director} and \texttt{historical-early-contract}, Naive scores 0 because the relevant documents were deleted; Vanilla scores 1 by retention accident; JAMES scores 1 by design.

\begin{table}[h]
\centering
\caption{S2 per-category R@10 (breakdown). Categories where Naive scores 0 are the ones the prior version was supersede-removed; categories where Vanilla scores high are the ones where the surviving stale documents happen to match the query.}
\label{tab:phase_b_cat}
\begin{tabular}{l c c c}
\toprule
\textbf{Category} & \textbf{Vanilla} & \textbf{Naive} & \textbf{JAMES} \\
\midrule
current-director & 1.000 & 1.000 & 1.000 \\
current-policy & 0.750 & 1.000 & 1.000 \\
current-project-lead & 1.000 & 1.000 & 1.000 \\
\textbf{historical-early-contract} & 1.000 & \textbf{0.000} & 1.000 \\
\textbf{historical-early-director} & 1.000 & \textbf{0.000} & 1.000 \\
\textbf{historical-mid-director} & 1.000 & 0.600 & 1.000 \\
historical-mid-policy & 0.500 & 0.000 & 0.500 \\
historical-mid-project-lead & 1.000 & 0.667 & 1.000 \\
never-stale-budget & 1.000 & 1.000 & 1.000 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Cross-model (v0.2.1)}
\label{sec:cross_model}

We extend Phase B's token-only retrieval with an optional LLM-grounded retrieval mode: top-20 from token-overlap, then LLM re-rank to top-$k$. We sweep four model families (\texttt{gemma4:e4b} 4B Google Gemma; \texttt{gemma3:12b} 12B Google Gemma; \texttt{mixtral:8x7b} 47B Mistral MoE; \texttt{claude-haiku-4-5} cloud Anthropic). Determinism: temperature $=0$, seed $=42$ where supported, format $=$ JSON.

\begin{table}[h]
\centering
\caption{S2 R@1 across four model families. The V $<$ N $<$ J rank order is preserved at every model size, from token-overlap baseline (no LLM) to cloud frontier. JAMES R@1 scales monotonically from 0.713 to 0.975 (claude); the validity-window contribution (J $-$ N) ranges +0.163 to +0.213 across models. Single-model fluke ruled out.}
\label{tab:cross_model}
\begin{tabular}{l c c c c c}
\toprule
\textbf{SUT} & \textbf{token} & \textbf{4B} & \textbf{12B} & \textbf{47B} & \textbf{claude} \\
\midrule
Vanilla & 0.225 & 0.488 & 0.400 & 0.375 & 0.6125 \\
Naive-supersede & 0.538 & 0.563 & 0.613 & 0.625 & 0.775 \\
\textbf{JAMES} & \textbf{0.713} & \textbf{0.725} & \textbf{0.775} & \textbf{0.838} & \textbf{0.975} \\
\midrule
J $-$ N gap & +0.175 & +0.163 & +0.163 & +0.213 & +0.200 \\
\bottomrule
\end{tabular}
\end{table}

\textbf{Cross-model finding.} The pre-registered v0.2.1 honest-tier ladder (Section~\ref{sec:cross_model_ladder}) required ``rank order V $<$ N $<$ J reproduces in all four models''. All four legs cleared; the cross-model gap structure is the publication-grade headline. The validity-window contribution (J $-$ N) is the stable cross-model metric, ranging from +0.163 to +0.213 across the four model families and +0.175 at the token baseline (a $\pm$0.03 band). The cross-SUT gap (J $-$ V) is non-monotonic across models because Vanilla absorbs heterogeneous lift from LLM re-rank: +0.488 at token, +0.237 at gemma4 4B, +0.375 at gemma3:12b, +0.463 at mxtral 47B, +0.363 at claude. We therefore frame the cross-model headline as ``J $-$ N gap stable'' rather than ``J $-$ V gap monotonic''.

\paragraph{Capability emergence is not monotonic.} An unexpected sub-finding: small models (gemma4 4B) regress on certain S1 cells where larger models lift. The pattern is not a single ``size threshold''; the LLM rerank value is a function of (task hardness, baseline strength, model capability). On hard S2 historical queries with weak Vanilla baseline, 4B alone lifts +0.263 R@1 over token; on easy S1 cells with strong JAMES baseline, 4B \emph{regresses} -0.033. This motivated the cross-bench reproducibility check in Section~\ref{sec:cross_bench}.

\subsection{HR axis (v0.2.4)}
\label{sec:hr_axis}

The Hallucination Resistance axis pipes the same LRB-S1 retrieved-and-answered cells through atomic-claim extraction (rule-based plus optional LLM augmentation) and two NLI verifiers (RoBERTa-large-MNLI primary, DeBERTa-v3-large-mnli-fever-anli-ling-wanli secondary). HR score $=$ fraction of atomic claims labelled \emph{entailment} (strict; neutral and contradiction both count against).

\begin{table}[h]
\centering
\caption{HR axis on LRB-S1 (T $=$ 12w, n $=$ 10), two NLI verifiers. The Vanilla / Naive / JAMES rank-order \textbf{V $<$ \{N, J\}} is preserved across both NLI verifiers and across two model families (12B, 47B). Mechanism: Vanilla's append-only context surfaces both old and new versions of superseded documents to the answer-generation LLM; the generated answer picks one version but the NLI verifier sees the full context (containing both) and labels the claim accordingly. Validity-filter SUTs (Naive and JAMES) feed a cleaner context where the answer aligns with the NLI premise.}
\label{tab:hr_cross_nli}
\begin{tabular}{l c c}
\toprule
\textbf{Cell} & \textbf{RoBERTa-MNLI} & \textbf{DeBERTa-v3 (ANLI)} \\
\midrule
gemma3:12b $\times$ Vanilla & 0.000 & 0.200 \\
gemma3:12b $\times$ Naive & 0.600 & 0.700 \\
gemma3:12b $\times$ JAMES & 0.600 & 0.700 \\
mxtral $\times$ Vanilla & 0.075 & --- \\
mxtral $\times$ Naive & 0.317 & --- \\
mxtral $\times$ JAMES & 0.283 & --- \\
\bottomrule
\end{tabular}
\end{table}

\textbf{HR honest framing.} At n $=$ 10 the JAMES vs.~Naive separation is not significant; the cross-verifier agreement on \textbf{V $<$ \{N, J\}} is the publishable finding. T5-XXL TRUE NLI Mixture and n $\geq$ 100 are deferred to v0.2 publication tier (operator-attended).

\subsection{Cross-bench reproducibility (MuSiQue)}
\label{sec:cross_bench}

To verify that LRB's SUT differentiation \emph{is} task-specific (lifecycle queries) and \emph{is not} task-specific (generic multi-hop reasoning), we re-run the three SUTs on MuSiQue-Ans dev~\cite{trivedi2022musique} with the LRB \texttt{answer\_gen} pipeline (\texttt{retrieve\_at} top-5 / 20-paragraph corpus per query / SQuAD-norm EM/F1 / \texttt{is\_supporting} recall).

\begin{table}[h]
\centering
\caption{MuSiQue-Ans dev n $=$ 20 $\times$ gemma3:12b. All three SUTs produce cell-by-cell identical EM, F1 differs by 0.014 (paragraph-order LLM non-determinism within noise), support\_recall identical. Closed-book multi-hop queries have no INGEST history, no SUPERSEDE, no validity-window relevance: the lifecycle mechanisms are no-op, and the three SUTs are equivalent by construction.}
\label{tab:musique_cross}
\begin{tabular}{l c c c}
\toprule
\textbf{SUT} & \textbf{EM} & \textbf{F1} & \textbf{support\_recall} \\
\midrule
Vanilla & 0.200 & 0.3002 & 0.825 \\
Naive-supersede & 0.200 & 0.3002 & 0.825 \\
JAMES & 0.200 & 0.2863 & 0.825 \\
\bottomrule
\end{tabular}
\end{table}

This is the negative-case cross-bench result and a key honest framing for the paper: the LRB gap structure is task-specific to lifecycle / time-travel queries; on benches that have no lifecycle axis, the three SUTs are equivalent. This is a feature of the LRB framing, not a limitation: it certifies that the gap structure measured in Section~\ref{sec:phase_b} is not an artefact of the JAMES adapter implementation.

\paragraph{Improvement loop on MuSiQue.} On user request we ran a 5-variant improvement loop (\texttt{k} $\in \{5, 10, 20\}$, optional LLM-rerank, optional chain-of-thought prompt). Best lever: \texttt{k} $=$ 20 (all 20 paragraphs fed to the LLM) lifts EM from 0.150 to 0.200 ($+0.05$); LLM rerank lifts F1 modestly without moving EM; CoT \emph{regresses} EM to 0.000 because 12B generates less specific answers under the step-by-step prompt; \texttt{k} $=$ 10 \emph{also} regresses EM (mid-rank distractors steal LLM attention). The \texttt{k} $=$ 20 lift is general-model (retrieval bottleneck removal) and \emph{not} a JAMES-specific lift; Section~\ref{sec:cross_bench} verifies SUT-equivalence at \texttt{k} $=$ 20.

\subsection{Cross-model honest-tier ladder}
\label{sec:cross_model_ladder}

The v0.2.1 pre-registration locked four leg conditions for the publication-tier landing:

\begin{enumerate}[leftmargin=*]
  \item \textbf{leg 1} Rank V $<$ N $<$ J reproduces on the smallest model (gemma4 4B) --- cleared (+0.075 / +0.163 / +0.237 deltas for $N{-}V$ / $J{-}N$ / $J{-}V$).
  \item \textbf{leg 2} Reproduces on gemma3:12b --- cleared (+0.213 / +0.163 / +0.375 for $N{-}V$ / $J{-}N$ / $J{-}V$).
  \item \textbf{leg 3} Reproduces on mxtral 47B --- cleared (+0.250 / +0.213 / +0.463 for $N{-}V$ / $J{-}N$ / $J{-}V$).
  \item \textbf{leg 4} Reproduces on claude-haiku-4-5 cloud --- cleared (+0.163 / +0.200 / +0.363 for $N{-}V$ / $J{-}N$ / $J{-}V$).
\end{enumerate}

All four legs cleared; the publication-tier landing is preserved across the model spectrum and is not a single-model artefact.

\subsection{Cross-scale reproducibility (v0.2.3 S3)}
\label{sec:cross_scale}

The S2 fixture is a hand-curated $\approx$200-doc city-operations corpus. To rule out the V $<$ N $<$ J ordering being an artefact of that specific document count or hand-curated vocabulary, we extended LRB with a deterministic programmatic generator (\texttt{build\_lrb\_scenario\_s3.py}) that produces three scale presets schema-identical to S2: \emph{smoke} (100 docs, 282 events, 100 queries), \emph{dev} (300 docs, 1.2k events, 300 queries), and \emph{publication} (1000 docs, 5.6k events, 1000 queries). The generator uses templated vocabulary primitives (20 dept adjectives, 10 domains, 50 first names, 40 last names, 30 contract domains, 7 contract types) so the publication-tier 1000-doc fixture has no hand-curated bottleneck. All three fixtures are SHA-pinned (operators regenerate byte-identically).

Phase B (time-travel) measurement of all three scale presets through the deterministic token-mode driver yields the following 4-point ladder:

\begin{center}
\begin{tabular}{lccc|c}
\toprule
Scenario & V R@1 & N R@1 & J R@1 & V $<$ N $<$ J \\
\midrule
S2 token (frozen)     & 0.225 & 0.538 & 0.713 & yes \\
S3 smoke              & 0.510 & 0.730 & 0.930 & yes \\
S3 dev                & 0.530 & 0.737 & 0.913 & yes \\
\textbf{S3 publication} & \textbf{0.502} & \textbf{0.721} & \textbf{0.845} & \textbf{yes} \\
\bottomrule
\end{tabular}
\end{center}

The R@1 V $<$ N $<$ J inequality is preserved at every scale point across the 12.5$\times$ scale range, and the JAMES $-$ Naive gap remains above $+$0.10 at every point (+0.175 / +0.200 / +0.176 / +0.124). Absolute magnitudes are scenario-sensitive: the synthetic vocabularies have lower retrieval ambiguity than the hand-curated S2 city-operations text, so all three SUTs achieve higher absolute R@1 on S3. Cross-scenario claims are therefore limited to ordering and gap structure, not absolute magnitude. The pre-registration locked this verdict matrix before measurement; the result doc \texttt{lrb-v023-s3-publication-scale-results-2026-06-12.md} pins the full per-category breakdown and the post-hoc S3.1 contract-vocabulary fix that closed a pre-S3.1 measurement artefact (\texttt{current-contract R@10}$=$0 across all SUTs from single-template title cluster collapse).

\section{Discussion}
\label{sec:discussion}

\paragraph{The validity-window mechanism is the publishable contribution.} Comparing the Phase A null result (Naive $\equiv$ JAMES) to the Phase B headline (V $<$ N $<$ J on R@1), the differentiator is the time-travel axis: on current-state-only queries, supersede-as-remove is sufficient; on mixed time-travel queries, supersede-as-remove loses information that is retrievable from the validity-window architecture. The cross-model robustness check (Section~\ref{sec:cross_model}) rules out the V $<$ N $<$ J rank-order being a single-model artefact; the cross-bench check (Section~\ref{sec:cross_bench}) rules out the gap being an implementation artefact of the JAMES adapter.

\paragraph{What LRB does \emph{not} claim.} LRB does not claim that the audit-native validity-window architecture is novel: ActiveGraph~\cite{nakajima2026activegraph} demonstrates the same architectural class independently, and lifecycle-aware retrieval is well-established in industry data warehouses. The contribution of this paper is the benchmark, the lifecycle-versus-current decomposition, and the cross-model + cross-bench validation pipeline. It also does not claim that JAMES is better at \emph{multi-hop reasoning}: Section~\ref{sec:cross_bench} explicitly verifies the opposite (3-SUT identical on MuSiQue); the LRB axes are retrieval-side, not reasoning-side.

\paragraph{Relationship to RAB.} RAB~\cite{seo2026rab} measures \emph{the audit log a system exports} (AC / RF / PC); LRB measures \emph{the retrieval quality the system delivers}. The two benchmarks share submission discipline (pre-registration, deterministic scoring, gap-table headline) and a research framing (operationalising audit + lifecycle axes in measurable benchmarks). They are intentionally orthogonal in axis: RAB headline says nothing about retrieval, LRB headline says nothing about audit. JAMES happens to be audit-native, but LRB does not require an audit-native SUT --- a Vanilla append-only RAG is a valid floor (Baseline-0) and the gap is the headline, not the JAMES score.

\paragraph{The HR axis is a v0.2.4 extension, not the core LRB contract.} The HR axis touches an LLM at the scoring step (NLI verifier classification). This is the only LRB axis that does so. The retrieval-side axes (Sections~\ref{sec:phase_a}, \ref{sec:phase_b}, \ref{sec:cross_model}) remain LLM-judge-free at the scoring layer. The cross-NLI agreement protocol (RoBERTa $+$ DeBERTa argmax classification) was locked in the v0.2.4 pre-registration to mitigate single-verifier artefacts. T5-XXL TRUE NLI Mixture (the ALCE-standard verifier~\cite{gao2023alce}) is deferred to v0.2 publication tier; the present results use only RoBERTa-large-MNLI and DeBERTa-v3.

\paragraph{Pattern robustness vs.~magnitude sensitivity.} The cross-scale ladder (Section~\ref{sec:cross_scale}) demonstrates a separation between two kinds of claim: the V $<$ N $<$ J \emph{ordering} and the JAMES $-$ Naive \emph{gap structure} survive a 12.5$\times$ scale jump (S2 N=80 $\to$ S3 publication N=1000) and a vocabulary swap (hand-curated city-operations $\to$ programmatic 30-domain templates), while the absolute R@1 magnitudes do not. We read this as evidence that the validity-window mechanism's contribution is a structural property of the architecture, not a coincidence of one corpus size or vocabulary distribution; correspondingly, we frame the cross-scenario result as a pattern-and-gap finding and explicitly do not claim cross-scenario magnitude propagation.

\section{Limitations}
\label{sec:limitations}

\begin{enumerate}[leftmargin=*]
  \item \textbf{Synthetic fixtures.} S1, S2, and S3 all use deterministic synthetic city-operations corpora. Real enterprise corpora may exhibit different lifecycle event distributions (e.g.~UPDATE-heavy financial filings vs.~SUPERSEDE-heavy policy documents). The cross-scale check (\S\ref{sec:cross_scale}) addresses scale-attribution within the same domain class; cross-domain generalisation remains future work.
  \item \textbf{Single audit-native SUT.} The JAMES adapter is the only audit-native point in the gap table. ActiveGraph~\cite{nakajima2026activegraph} adapter would strengthen the within-class comparison; we list this as future work \S\ref{sec:future}.
  \item \textbf{Token-overlap retrieval baseline.} Vanilla, Naive, and JAMES all use the same TF-IDF base retriever in this paper. A learned dense retriever (e.g.~ColBERT, DPR) could change the absolute scores but is orthogonal to the lifecycle/time-travel axis we measure.
  \item \textbf{HR axis n.} v0.2.4 HR measurements use n=10 LRB-S1 cells. Full sweep (n=100+) is in the operator-attended queue.
  \item \textbf{TimeQA and TempReason measurements absent.} The Track C cross-bench measurements (TimeQA primary, TempReason secondary) require data download and license confirmation; we list these as the next milestones and report the MuSiQue cross-bench reproducibility check as the only currently-completed external-bench validation.
  \item \textbf{Single-author measurement.} All measurements were performed in a single repository by a single author. External reproduction is invited; the artefacts are committed and deterministic.
\end{enumerate}

\section{Future Work}
\label{sec:future}

\begin{enumerate}[leftmargin=*]
  \item \textbf{ActiveGraph adapter.} Within-class second audit-native SUT; mitigates the single-audit-native-row concern \S\ref{sec:limitations}.2.
  \item \textbf{TimeQA and TempReason measurement.} Track C primary and secondary benches once operator-action license / download gate clears.
  \item \textbf{HR full sweep (n=100+).} v0.2.4 full sweep across LRB-S1 + LRB-S2 + Track C benches; cross-NLI agreement at publication scale.
  \item \textbf{LLM-grounded S3 publication run.} Section~\ref{sec:cross_scale} reports token-mode only; an LLM-grounded re-run (claude / mxtral / gemma3:12b / gemma4:e4b) at N$=$1000 would extend the v0.2.1 cross-model leg-clear to publication-scale evidence.
  \item \textbf{Microsoft GraphRAG SUT.} Fourth SUT for cross-architecture gap structure.
  \item \textbf{Korean / cross-lingual scenario.} LRB v0.3 candidate; addresses the English-only fixture limitation.
\end{enumerate}

\section*{Data and Code Availability}

The LRB scenario generators (S1 / S2 / S3), the 3-SUT adapter implementations (Vanilla / Naive-supersede / JAMES validity-window), the deterministic scorer, the cross-model and cross-scale runners, the v0.2.4 HR NLI adapter, the eight pre-registration LOCK documents, and every \texttt{result.json} + \texttt{bench.jsonl} measurement artefact reported in this paper are archived at Zenodo DOI \href{https://doi.org/10.5281/zenodo.20652679}{\texttt{10.5281/zenodo.20652679}} (PROJECT~JAMES v0.4.4, 2026-06-12). The same DOI bundles the sibling RAB v0.1.1 software (the RAB axes operationalising EU AI Act Art.~10/12/19) so the two benchmarks can be cited from a single data-availability anchor. Every scenario fixture is byte-deterministic given the generator commit; LLM-grounded mode inherits the $\approx$10pp claude-API reproducibility band documented separately in \texttt{docs/research/lrb-v021-s2-vanilla-reproducibility-band-2026-06-12.md}. External reproduction is invited.

\section*{Acknowledgements}

The author thanks the open-source maintainers of Ollama, HuggingFace Transformers, the RoBERTa-large-MNLI checkpoint (Facebook AI Research) and the DeBERTa-v3-large-mnli-fever-anli-ling-wanli checkpoint (M.~Laurer) for the verifiers used in the v0.2.4 HR axis cross-NLI agreement protocol, and the authors of MuSiQue (Trivedi et~al.~2022) for releasing the dev split used as the cross-bench reproducibility check in \S\ref{sec:cross_bench}. The honest-framing self-correction reported in \S\ref{sec:cross_scale} (the S3.1 contract-vocabulary fix that retracted a pre-S3.1 over-tight verdict) is the strongest applied evidence of the project's measurement-side-artefact rule and was caught by per-category audit before any external reviewer reading.

\bibliographystyle{plain}
\bibliography{refs}

\end{document}
