# Context engineering is replacing prompt engineering as AI's core discipline

**Context engineering** — the practice of designing dynamic systems that deliver the right information, tools, and state to an LLM at the right time — has rapidly emerged as the defining skill for building production AI agents. Where prompt engineering focused on crafting a single instruction string, context engineering treats the model's entire input as an engineered system: retrieved documents, tool definitions, conversation history, memory, and structured state, all assembled programmatically at runtime. The term gained explosive traction in mid-2025 after endorsements from Andrej Karpathy, Shopify CEO Tobi Lütke, and practitioners at Anthropic and Google DeepMind, but the underlying discipline reflects hard-won lessons from teams building agentic systems throughout 2024–2025. The shift matters because, as Philipp Schmid of Google DeepMind put it: **"Most agent failures are not model failures anymore — they are context failures."**

## A single tweet crystallized what builders already knew

The term "context engineering" is widely attributed to **Walden Yan**, co-founder of Cognition (maker of the Devin coding agent), who drew on lessons from building autonomous agents to distinguish between manually crafting prompts and dynamically assembling context at scale. On June 18, 2025, **Tobi Lütke** posted on X: "I really like the term 'context engineering' over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM." One week later, **Andrej Karpathy** amplified the concept with a definition that became canonical: "Context engineering is the delicate art and science of filling the context window with just the right information for the next step." He enumerated its components — task descriptions, few-shot examples, RAG, multimodal data, tools, state, history, compaction — and noted that "doing this well is highly non-trivial."

The term resonated because it solved a naming problem. Simon Willison observed that most people's inferred definition of "prompt engineering" was "a laughably pretentious term for typing things into a chatbot," while the inferred definition of "context engineering" would land much closer to the actual complexity involved. Within weeks, Anthropic published a comprehensive engineering guide, LangChain formalized a four-strategy framework, and the Manus team shared production lessons from serving millions of users. By late 2025, Gartner was recommending that enterprises appoint context engineering leads.

## The anatomy of context: everything the model sees

Context engineering operates on a fundamental insight: **an LLM's behavior is determined not just by its instructions but by the totality of tokens it receives**. Karpathy's widely adopted analogy frames this precisely — the LLM is a CPU, the context window is RAM, and the context engineer's job is to act as an operating system, loading working memory with exactly the right code and data for each computational step.

Philipp Schmid's formalization identifies the components that context engineering must orchestrate. System instructions and behavioral rules form the stable foundation. The user's immediate prompt provides the task. Short-term memory preserves recent conversation turns. Long-term memory stores persistent knowledge across sessions. Retrieved information from RAG pipelines injects external knowledge. Tool definitions expose available capabilities. Tool responses feed execution results back into reasoning. Structured output schemas constrain response format. And global state — scratchpads, task trackers, environment observations — maintains coherence across agent steps.

Anthropic introduced the concept of an **"attention budget"** to formalize the core constraint: context windows are finite, and their utility follows diminishing returns. As tokens accumulate, the model's ability to accurately recall information degrades — a phenomenon they termed **"context rot."** Good context engineering therefore means finding the smallest possible set of high-signal tokens, not cramming in everything available. This reframes the discipline from "how to write good prompts" to "how to engineer an information system under tight computational constraints."

## How context engineering differs from prompt engineering

The distinction is both conceptual and structural. Prompt engineering operates within a single input-output pair — it concerns what you say to the model. Context engineering concerns everything the model sees when you say it: the full information architecture assembled dynamically at runtime. Prompt engineering is manual and static; context engineering is systematic and dynamic. Prompt engineering is crafting a function; context engineering is designing the software architecture.

Concretely, a prompt engineer writes and iterates on instruction text, designs few-shot examples, and optimizes phrasing. A context engineer builds retrieval pipelines, architects memory systems, manages token budgets across competing information sources, designs multi-agent context handoff protocols, monitors KV-cache hit rates, and implements compression strategies — all while ensuring the assembled context enables the model to succeed. Anthropic's official position frames context engineering as **"the natural progression of prompt engineering,"** with prompt engineering as a proper subset focused exclusively on the instruction layer.

The practical gap becomes obvious in production. BigData Boutique, citing LangChain's 2025 State of Agent Engineering report, notes that **57% of organizations now have AI agents in production, but 32% cite quality as their top barrier** — and most quality failures trace to poor context management, not model limitations. Drew Breunig identified four distinct context failure modes that prompt engineering cannot address: poisoning (wrong information injected), distraction (irrelevant context drowning relevant signals), confusion (contradictory information), and clash (conflicting instructions from different sources). Each requires systems-level solutions, not better wording.

## Ten techniques that define the practitioner's toolkit

The technical substrate of context engineering spans retrieval, memory, compression, caching, and orchestration. LangChain's Lance Martin organized these into four strategic buckets — **Write, Select, Compress, and Isolate** — that map cleanly onto the practitioner's daily work.

**Retrieval-Augmented Generation** remains the primary "Select" strategy. Production RAG pipelines for agents go well beyond basic embedding search: they combine keyword, semantic, and knowledge-graph retrieval in hybrid configurations, apply cross-encoder re-ranking to prioritize the most relevant chunks, and increasingly use "agentic RAG" patterns where the agent itself decides when and what to retrieve rather than retrieving on every turn. Context ordering matters significantly — the "lost in the middle" phenomenon means models attend better to information at the beginning and end of context, so retrieved documents must be strategically positioned.

**Memory management** has evolved into a sophisticated subsystem. The Letta (formerly MemGPT) framework pioneered a tiered architecture mirroring operating system memory hierarchies: a message buffer for immediate context, core memory blocks the agent can read and write via tool calls, recall memory for searchable conversation history, and archival memory for long-term knowledge storage. OpenAI's production patterns classify memory signals by stability (preferences that rarely change), drift (gradual evolution), and contextual variance (situation-dependent preferences), enabling intelligent memory updates rather than naive fact accumulation. Amazon's Bedrock AgentCore, Mem0, and Zep provide managed memory layers with automatic extraction and hybrid retrieval.

**Tool use context** presents unique challenges as the MCP (Model Context Protocol) ecosystem now exceeds **10,000 active public servers**. Tool definitions consume significant tokens, and the Manus team discovered that "your heavily armed agent gets dumber" when too many tools flood the action space. Their solution — **masking unavailable tools rather than removing them** — preserves KV-cache efficiency while controlling the action space. Tool result management requires compression of verbose API responses before re-injection, and self-healing patterns feed error messages back into context so agents can recover from failed tool calls.

**Conversation history management** uses sliding windows, LLM-driven summarization, and hybrid approaches. JetBrains Research (December 2025) found that combining observation masking with summarization reduced costs by **7–11%** while increasing task success rates by approximately 2.6% on SWE-bench benchmarks. The critical insight is that summaries act as "clean rooms" that can correct or omit prior mistakes, preventing context poisoning from propagating forward — but even minor hallucinations in summaries compound over time, demanding careful quality control.

**Context compression** operates at multiple levels. Hard prompt methods like LLMLingua filter low-information tokens to achieve up to **20x compression**. Soft prompt methods encode context into continuous embeddings for up to 480x compression ratios, though with fidelity tradeoffs. The ACON framework (October 2025) demonstrated **26–54% reduction in peak tokens** while preserving task success on multi-step agentic benchmarks. Factory.ai's evaluation of production compression strategies found that structured summarization — with explicit sections for different information types — scored highest at 3.70/5 because "structure forces preservation" of critical details.

**Caching strategies** provide massive production leverage. Prompt caching reuses previously computed KV tensors for identical prompt prefixes, yielding a **10x cost reduction** (cached vs. uncached tokens on Anthropic's API) and **64–65% latency improvements** on stable prefixes. The Manus team considers KV-cache hit rate "the single most important metric for a production-stage AI agent." Maximizing cache hits requires architectural discipline: front-loading stable content (system prompts, agent identity) as a contiguous prefix, appending dynamic content at the end, and using append-only context patterns rather than modifying earlier content.

**Dynamic context injection** selects what to include based on the current task at runtime. Rather than pre-loading everything, agents maintain lightweight identifiers and load detailed data on demand using tools — Anthropic calls this "just-in-time context." **Multi-agent context passing** requires careful scoping to prevent context explosion; Google's ADK framework controls this with an `include_contents` parameter that determines whether sub-agents inherit full or empty context from their callers. Multi-agent systems can consume **up to 15x more tokens** than standard interactions, making context isolation essential.

## Why agents specifically demand context engineering

The transition from chatbots to agents created qualitative changes in context management that prompt engineering could not absorb. Agents operate in iterative tool-use loops where each step adds actions and observations to a growing context. Manus reports an average **input-to-output token ratio of 100:1** — the context grows relentlessly while output remains short. This asymmetry means context management dominates both cost and quality.

Anthropic articulates the core challenge: "An agent running in a loop generates more and more data that could be relevant for the next turn of inference, and this information must be cyclically refined. Context engineering is the art and science of curating what will go into the limited context window from that constantly evolving universe of possible information." The Manus team rebuilt their agent framework **four times** searching for better context architectures — a process they called "Stochastic Graduate Descent."

The failure modes are agent-specific. Context rot degrades recall accuracy as tokens accumulate across long operation chains. Context poisoning causes hallucinations to propagate forward through subsequent reasoning steps. Tool overload floods the action space and reduces decision quality. Cross-session memory loss forces agents to re-learn user preferences and task context. None of these problems respond to better prompt wording — they require engineered systems with retrieval pipelines, compression triggers, memory hierarchies, and cache-aware architectures.

## Frameworks and mental models shaping the field

Several organizing frameworks have crystallized. Karpathy's **OS analogy** provides the foundational mental model: the context engineer as operating system designer, mediating between the LLM's computational capabilities and the information environment. LangChain's **Write/Select/Compress/Isolate taxonomy** offers an actionable strategic framework — every context engineering decision falls into persisting information externally, pulling relevant information in, reducing token count, or scoping context across agents.

Anthropic's **"right altitude" framework** for system prompts identifies the Goldilocks zone between brittle over-specification and vague under-specification. Dex Horthy's **12 Factor Agents** manifesto established foundational principles including "own your context window" and identified a performance "dumb zone" when more than **~40% of the context window** is consumed. The ACE (Agentic Context Engineering) research framework treats contexts as evolving "playbooks" that accumulate, reflect on, and curate strategies through structured iterations, achieving **+10.6% improvements** on agent benchmarks.

The Manus team's production lessons distill into engineering principles: treat KV-cache hit rate as your north star metric, use append-only context to preserve cache efficiency, mask rather than remove tools, and accept that context engineering remains "an experimental science" requiring iterative architecture search. Google's ADK separates Session (ground truth state) from Working Context (the compiled view sent to the LLM), enabling optimization without information loss. Together, these frameworks represent a maturing discipline with both theoretical depth and battle-tested production patterns.

## Conclusion

Context engineering represents a genuine paradigm shift in how practitioners build with LLMs — from crafting individual instructions to architecting dynamic information systems. The discipline emerged not from academic theory but from the concrete failures teams encountered when deploying agents in production: context rot, poisoning, tool overload, and memory loss that no amount of prompt iteration could fix. The convergence of endorsements from Karpathy, Lütke, Anthropic, and Google DeepMind in mid-2025 gave the field a name, but the underlying techniques — RAG, tiered memory, context compression, cache-aware architecture, multi-agent scoping — had been developing throughout 2024–2025 as the gap between chatbot-era prompt engineering and agent-era requirements became undeniable. The most consequential insight may be Schmid's: that **80% of agent failures stem from context problems, not model problems**. As models improve, the bottleneck shifts decisively from model capability to information architecture — making context engineering the primary lever for building AI systems that actually work.
