ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions

ContextEcho measures whether a frontier model's trained Assistant persona survives thousands of tool-using turns in real coding-agent work, and whether short interventions can restore the trained register.

Xianzhong Ding, Yangyang Yu, Changwei Liu, Bill Zhao

Paper Code Dataset Donate Session Demo
23frontier model targets
10model organizations
41,921per-cell evaluations
57accepted sessions (54 community + 3 founding)
47,350tracked dataset downloads
ContextEcho framework overview showing session snapshotting, persona probes, scoring, and mitigation.

ContextEcho snapshots real long-running coding-agent sessions, forks the conversation state, and runs a 25-probe identity suite without perturbing the donor's original session.

Demo

Watch drift and mitigation side by side

The same probe can produce different behavior late in a long session. The recorded demo shows the drift arm and the mitigated/control arm running side by side.

Left: drift without anchor. Right: mitigation with the A-anchor applied.

Run Interactive Demo Locally
Local demo command and notes

The public page shows a recorded demo. The interactive version streams fresh probes locally with your own API key.

python -m demo_live.server
# then open http://localhost:8765

Read the demo README

Overview

What ContextEcho Measures

The benchmark focuses on behavior drift in agentic-coding contexts: not whether a model can code, but whether its trained Assistant register survives long, tool-heavy work.

Benchmark

25-probe identity suite

ContextEcho probes surface style, task framing, refusal discipline, instruction following, and assistant register across session positions and model targets.

Long sessions

Real agentic-coding traces

The released corpus includes redacted donor sessions and per-cell JSON evaluations from long Claude Code and Codex CLI workflows.

Mitigation

Anchor recovery experiments

A short Assistant-register anchor is tested as a practical intervention, including persistence, size, and cross-target behavior.

Dataset

A living public dataset with local-first donation

The public release separates code and data, tracks donation lineage, and grows through a donor wizard that redacts on the donor machine before private maintainer review.

Open Dataset Dataset Card Contributors
12,337user turns across the 55 community ledger sessions
169context compactions across the 57 accepted sessions
31public contributors in the leaderboard
54promoted accepted donations in the v2 ledger

Pipeline

From local session to public benchmark row

The public tools are part of the benchmark, not a side project: they make the dataset auditable, extensible, and safer to grow.

1DiscoverThe browser wizard finds local Claude Code and Codex sessions.
2RedactPII, secrets, paths, and custom terms are removed locally.
3SubmitOnly redacted artifacts are uploaded through the relay.
4ReviewMaintainers inspect staging artifacts, lineage, and redaction checks.
5ReleaseAccepted sessions are promoted into the public dataset ledger.

Results

Persona drift appears across model families

General, not family-specific. The headline panel measures 23 targets from 10 organizations.
Compaction is not a reliable reset. Some post-compaction trajectories continue to show drift rather than recovery.
A short anchor helps. A roughly 110-token Assistant-register anchor restores the trained register across measured targets in the paper experiments.
ContextEcho headline forest plot showing drift gaps with confidence intervals across model targets.
Headline forest plot from the released reproduction artifact.

Privacy and ethics

ContextEcho studies assistant behavior, not donor personality

The donation path is designed around donor control. Raw local histories stay on the donor machine; public release artifacts are redacted, reviewed, and summarized without donor emails or donor-to-institution links.

Donor Privacy Data Use Policy

Public reporting boundary

ContextEcho reports aggregate dataset composition and assistant-behavior measurements. Donors can provide maintainer-visible contact fields while choosing public anonymous credit. The default donation mode is full redacted.

Reproduce

Every paper artifact has a command

The repository includes a claim-by-claim reproduction document, plotting scripts, analysis scripts, and idempotent experiment runners for recollecting cells when provider access is available.

Show reproduction commands 
make setup
make verify-pii
make fig2-forest
make figs-body
make figs-app

Resources

Paper, data, tools, and live workflows

PaperRead the arXiv preprint and cite the benchmark. GitHubRuntime, harness, plotting, and donation code. DatasetReleased redacted sessions and per-cell evaluations. DonateRun the local-first wizard and contribute a session. Maintainer ConsoleReview staging submissions and relay records. ReproductionCommands for figures, claims, and audits. ContributorsPublic credit and donation leaderboard. Live DemoSide-by-side drift demonstration from the repo.

BibTeX

Citation

Show BibTeX 
@article{ding2026contextecho,
  title={ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions},
  author={Ding, Xianzhong and Yu, Yangyang and Liu, Changwei and Zhao, Bill},
  journal={arXiv preprint arXiv:2605.24279},
  year={2026}
}