What Is Context Engineering? A Practical Guide for AI Agents

Context engineering is the practice of deciding what an AI model sees at every step: the instructions, tool definitions, retrieved documents, memory and conversation history in its context window, kept as small and as relevant as the task allows. It matters most for AI agents, which call tools in a loop and fill their own context with every result. This guide covers where the term came from, how long contexts fail, the techniques vendors ship today, tested code, an evaluation plan and a worked before-and-after example.

What is context engineering?

Anthropic's Applied AI team defines it as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts." Their post Effective context engineering for AI agents states the goal in one line: find "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome."

LangChain's Harrison Chase frames it as a system: "building dynamic systems to provide the right information and tools in the right format such that the LLM can plausibly accomplish the task." The key word is dynamic: on an agent's twentieth step, most of what the model reads was written by tools, retrievers and earlier turns, and in Anthropic's words that information "must be cyclically refined."

Where the term came from

None of the posts that popularized the phrase claims to have invented it. It gained traction over a few weeks in June 2025:

• 12 June 2025. In "Don't Build Multi-Agents", Walden Yan of Cognition called context engineering "effectively the #1 job of engineers building AI agents."
• 19 June 2025. Shopify CEO Tobi Lütke wrote on X: "I really like the term 'context engineering' over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM."
• 23 June 2025. Harrison Chase published "The rise of 'context engineering'" on the LangChain blog, with the definition quoted above.
• 25 June 2025. Andrej Karpathy posted "+1 for 'context engineering' over 'prompt engineering'", calling it "the delicate art and science of filling the context window with just the right information for the next step."
• 29 September 2025. Anthropic published its guide, which Anthropic's API documentation now cites when it explains why long contexts degrade.

The principles fit in a paragraph. Turning them into a token budget, memory layers and compaction rules for one specific agent is the hard part, covered module by module in Context Engineering and Memory for AI Agents: Beyond Prompting, from the anatomy of the context window to evaluating memory for reliability.

Context engineering vs prompt engineering

Prompt engineering is writing and organizing instructions. Context engineering covers everything that reaches the model, on every call. Anthropic calls it "the natural progression of prompt engineering"; Chase calls prompt engineering "a subset of context engineering."

Anthropic puts the difference in one sentence: "In contrast to the discrete task of writing a prompt, context engineering is iterative and the curation phase happens each time we decide what to pass to the model."

Prompt-writing skills still apply to the part of the context you write by hand, and our prompt engineering guide covers them from the basics. What changes is the diagnosis. For a failing agent, Chase suggests asking first: "Is it failing because you haven't given it the right information or tools? Or does it have all the right information and it just messed up?" In his view, "more often than not" the context is the cause, and no rewording of the system prompt fixes missing information.

What fills the context window

Every model call sees a stack of components, each with its own typical failure and its own first lever:

For the parts you write, Anthropic's advice is to aim system prompts at "the right altitude", "specific enough to guide behavior effectively, yet flexible enough to provide the model with strong heuristics", and to keep the toolset small: "If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better."

For everything else, Google's Agent Development Kit (ADK) team offers a useful mental model. In their December 2025 architecture post, context is "a compiled view over a richer stateful system": sessions, memory and artifacts are the sources, compiled into a working context for each call. Their principles double as a checklist: separate storage from presentation, build context through explicit, testable transformations, and scope by default, so that "every model call and sub-agent sees the minimum context required."

Why more context makes agents worse

Bigger windows did not remove the problem. All four current Claude models list a 1M-token context window, and Anthropic's documentation still warns that "more context isn't automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot." The engineering post traces it to attention, which creates "n² pairwise relationships for n tokens", and describes the effect as "a performance gradient rather than a hard cliff."

The clearest public measurement is Chroma's July 2025 report, Context Rot: How Increasing Input Tokens Impacts LLM Performance, which evaluated 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3. When models answered the same LongMemEval questions from a focused prompt of about 300 tokens and from the full input of about 113k tokens, the report found: "Across all models, we see significantly higher performance on focused prompts compared to full prompts." The relevant information was in both versions; the irrelevant context alone made it harder to use.

Four ways a context goes wrong

In June 2025 Drew Breunig named four specific failure modes, a taxonomy LangChain reused in its July 2025 guide. Each has a primary source behind it, and a symptom you can spot in a trace:

• Context poisoning is "when a hallucination or other error makes it into the context, where it is repeatedly referenced." Google's Gemini 2.5 technical report describes it in Gemini Plays Pokémon, an agent an independent developer built on Gemini 2.5 Pro, where parts of the context "(goals, summary) are 'poisoned' with misinformation about the game state, which can often take a very long time to undo." Symptom: the agent keeps acting on a "fact" no tool ever returned. First fix: correct the note or summary that carries it.
• Context distraction is "when a context grows so long that the model over-focuses on the context, neglecting what it learned during training." The report also observed that "as the context grew significantly beyond 100k tokens, the agent showed a tendency toward favoring repeating actions from its vast history rather than synthesizing novel plans," and calls the observation anecdotal. Symptom: repeated actions instead of a new plan. First fix: compaction, structured notes, a fresh context per sub-task.
• Context confusion is "when superfluous content in the context is used by the model to generate a low-quality response." Too many tools is the classic case: Anthropic's tool search documentation says Claude's "ability to pick the right tool degrades once you exceed 30–50 available tools." Symptom: an irrelevant tool call or document. First fix: fewer tools, tool search, tighter retrieval.
• Context clash is "when you accrue new information and tools in your context that conflicts with other information in the context." Breunig illustrates it with the paper LLMs Get Lost In Multi-Turn Conversation (May 2025), which found "an average drop of 39% across six generation tasks" when a task arrived over several turns instead of one. Symptom: an old answer or an outdated policy mixed into the reply. First fix: remove superseded content and restate the current goal.

The core techniques: write, select, compress, isolate

LangChain's July 2025 guide grouped the techniques into four buckets: write context ("saving it outside the context window"), select context ("pulling it into the context window"), compress context ("retaining only the tokens required to perform a task") and isolate context ("splitting it up to help an agent perform a task"). Here is what each looks like with the features vendors document today.

Select: retrieve just in time

Classic RAG retrieves before the call. Anthropic describes agents moving to a "just in time" approach: keep "lightweight identifiers (file paths, stored queries, web links, etc.)" and load data with tools only when a step needs it. Claude Code is the hybrid example: CLAUDE.md files are loaded up front, while glob and grep fetch files on demand. The cost is speed, since "runtime exploration is slower than retrieving pre-computed data." Google's ADK applies the same handle pattern to large artifacts: by default the agent sees a name and a summary, and loads the content with a tool only when it needs it.

Tools need selecting too. Anthropic's tool search documentation estimates that a typical setup with five servers (GitHub, Slack, Sentry, Grafana and Splunk) can consume about 55k tokens of definitions before Claude does any work, and that tool search "typically reduces this by over 85 percent." You mark rarely used tools with defer_loading: true and let the model search for them.

When retrieval is your main source of context, chunking, hybrid search, re-ranking and retrieval evaluation decide how much of what you select is signal. They are covered in depth in RAG: Retrieval-Augmented Generation in Practice.

Compress: shape tool output before it enters the context

A tool result rides along on every later step, so the tool layer is the cheapest place to save tokens. Anthropic's post Writing effective tools for agents — with agents recommends "pagination, range selection, filtering, and/or truncation with sensible default parameter values" (Claude Code's documentation caps MCP tool output at 25,000 tokens by default, adjustable through MAX_MCP_OUTPUT_TOKENS), meaningful fields instead of "low-level technical identifiers", and, when you truncate, to "steer agents with helpful instructions." Here is that advice as a function you can put between any API and the model (plain Python, no SDK):

import json


def shape_tool_result(items: list[dict], fields: tuple[str, ...],
                      limit: int = 10, max_chars: int = 4000) -> str:
    """Turn a raw API payload into a small, readable tool result for the model."""
    rows = [{f: item.get(f) for f in fields} for item in items[:limit]]
    while rows and len(json.dumps(rows, sort_keys=True)) > max_chars:
        rows.pop()  # drop whole rows so the JSON stays valid
    text = json.dumps(rows, sort_keys=True)
    if len(rows) < len(items):
        text += (f"\n[Showing {len(rows)} of {len(items)} results. Narrow the query "
                 "or request the next page; call get_order(order_id) for one full record.]")
    return text

It keeps only the fields the next decision needs, drops whole rows so the model never sees broken JSON, tells the agent how to get more instead of hiding data silently, and uses sort_keys=True so the output stays byte-for-byte stable for caching. Characters are a rough proxy for tokens; use your provider's token counter when you need precision. The Manus team adds one rule for anything you drop: make it restorable, because "the content of a web page can be dropped from the context as long as the URL is preserved."

Compress: clear, trim and compact the history

Even shaped results pile up. Three tools shrink the history, lightest first:

1. Clear old tool results, which Anthropic calls "one of the safest lightest touch forms of compaction." On the Claude API this is the clear_tool_uses_20250919 context-editing strategy, in beta. By default it starts at 100,000 input tokens and keeps the three most recent tool uses; clear_at_least makes each clearing large enough to justify breaking the prompt cache.
2. Trim old turns. OpenAI's Agents SDK cookbook on session memory sums up the trade-off: trimming is cheapest but forgets long-range context ("context loss"), while summarization keeps it but risks "context distortion/poisoning", so log the summaries and evaluate them.
3. Compact: summarize the history and continue from the summary. Anthropic says compaction "typically serves as the first lever in context engineering to drive better long-term coherence."

Anthropic's and OpenAI's APIs now both compact on the server. The Claude API offers compaction at a token threshold (a compact_20260112 edit with a default trigger of 150,000 input tokens and a minimum of 50,000) and compaction on demand, both in beta; the documentation recommends on-demand compaction wherever it is available. OpenAI's Responses API compacts when the context crosses a compact_threshold set in context_management, or through a standalone /responses/compact endpoint, as its compaction guide describes. Here is the threshold mode, the shortest to show, with the Anthropic Python SDK (version 2.4.0, current as of November 2026):

import anthropic

client = anthropic.Anthropic()

SUMMARY_INSTRUCTIONS = (
    "Summarize this support conversation so another agent can continue it. "
    "Keep the customer's goal, order IDs, facts confirmed by tools, actions taken "
    "and their results, anything promised to the customer, and open questions. "
    "If an earlier statement turned out to be wrong, record the correction. "
    "Drop raw tool output that is no longer needed."
)

messages = [{"role": "user", "content": "My blender arrived broken. Order 48213."}]

response = client.beta.messages.create(
    betas=["compact-2026-01-12"],
    model="claude-opus-5-5",
    max_tokens=4096,
    messages=messages,
    context_management={
        "edits": [
            {
                "type": "compact_20260112",
                "trigger": {"type": "input_tokens", "value": 60000},
                "instructions": SUMMARY_INSTRUCTIONS,
            }
        ]
    },
)

# Append the whole response, including any compaction block, then continue the loop
messages.append({"role": "assistant", "content": response.content})

Custom instructions "don't supplement the default prompt. They replace it completely," so ask for everything the next step needs; on later requests the API drops everything before the compaction block by itself. One vendor difference matters for testing: Claude's compaction block carries summary text you can log and grade, while OpenAI's compaction item is "opaque and not intended to be human-interpretable," so there you judge compaction by the agent's later behavior.

For the summary prompt, Anthropic's advice is to start by "maximizing recall", then "improve precision by eliminating superfluous content." Claude Code's own compaction keeps "architectural decisions, unresolved bugs, and implementation details" and resumes with "the five most recently accessed files"; its /context command shows what is filling the window.

Write: scratchpads, structured notes and memory

Compaction decides after the fact what survives; notes let the agent decide while it works. Anthropic describes "structured note-taking, or agentic memory", in which the agent "regularly writes notes persisted to memory outside of the context window", like Claude Code's to-do list or a NOTES.md file. In Anthropic's example of Claude playing Pokémon, the agent read its own notes after each context reset and carried on. Manus keeps a todo.md and rewrites it through tasks that average "around 50 tool calls", which pushes "the global plan into the model's recent attention span."

A support agent's case file, rewritten after every tool call, might look like this:

# Case 48213
Goal: refund for a blender delivered damaged
Confirmed by tools:
- Delivered 2026-10-28 (get_order)
- Refund request RF-7731 created, status: pending (create_refund)
Promised to customer: an update within 24 hours
Corrections: refund NOT issued yet (an earlier reply said it was)
Open questions: does the seller's 30-day return window apply?

The "Corrections" line reconciles two pieces of advice. Manus recommends that you "leave the wrong turns in the context", because a visible failure steers the model away from repeating it. Poisoning is different: a false belief, not a recorded failure. Keep the evidence of what happened; overwrite claims about what is true.

For memory across sessions, the Claude API's memory tool (memory_20250818, now listed as stable) gives Claude a /memories directory whose file operations your application runs against storage you control, and "Claude automatically checks its memory directory before starting a task." Your handler must reject any path outside /memories, since the paths come from the model.

Isolate: give sub-agents their own context

A sub-agent works in a clean window and hands back a summary. In Anthropic's description, it "might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens)." Anthropic's multi-agent research system, a Claude Opus 4 lead with Claude Sonnet 4 sub-agents, "outperformed single-agent Claude Opus 4 by 90.2%" on an internal research eval, but "multi-agent systems use about 15× more tokens than chats."

Cognition argues the other side with two principles: "Share context, and share full agent traces, not just individual messages," and "Actions carry implicit decisions, and conflicting decisions carry bad results." In its example, two sub-agents building parts of a Flappy Bird clone made choices that did not fit together. The positions meet in practice: isolate read-heavy, independent work (research, search, log analysis), and keep decisions that must agree in one thread. Google's ADK makes the scope explicit: in its "agents as tools" pattern the callee "sees only the specific instructions and necessary artifacts—no history," and an include_contents setting decides how much history a sub-agent inherits on a handoff.

Choosing between these architectures runs through AI Agents: Architecting and Automating Autonomous Systems, and the production side (the agent loop, guardrails, observability) is in our guide to building AI agents that work in production.

Keep the prefix stable: context engineering and caching

Every technique above edits the context, and an edit costs money when it breaks the prompt cache. Manus's Yichao "Peak" Ji argues that "the KV-cache hit rate is the single most important metric for a production-stage AI agent", because agents read far more than they write (in Manus, "the average input-to-output token ratio is around 100:1") and "even a single-token difference can invalidate the cache from that token onward." The practical rules:

• Put stable content (instructions, tool definitions) first and volatile content last; a timestamp at the top of the system prompt is the classic mistake.
• Keep the context append-only and the serialization deterministic.
• Do not add or remove tool definitions mid-task. Manus masks unavailable tools during decoding instead, and Anthropic's tool search appends discovered tools inline, so "the prefix is untouched, so prompt caching is preserved."
• Clear and compact in batches rather than on every turn, since clearing tool results "invalidates cached prompt prefixes when content is cleared."

How to measure context engineering with evals

Context changes can fail in both directions: a summary that drops an order ID, or a retrieval cutoff that hides the one document that mattered. Measure them like any other change. Anthropic's guide Demystifying evals for AI agents (January 2026) starts small: "20-50 simple tasks drawn from real failures is a great start." It separates the transcript (everything the agent did) from the outcome (the final state of the environment), and notes that "latency, token usage, cost per task, and error rates can be tracked on a static bank of tasks." For context engineering, track:

1. Outcome success, graded against the real end state (was the refund created?), not the agent's claim.
2. Tokens and cost per successful task, not per call. Saving tokens per call while adding turns can cost more.
3. Consistency. With "a 75% per-trial success rate," the same guide notes, the chance that all three of three trials pass is "(0.75)³ ≈ 42%," so run each task several times and report pass^k.
4. Compaction recall. List the facts a long task needs after compaction (IDs, decisions, promises) and check that the summary keeps them.
5. Degradation by length. Bucket results by context size or turn count; context rot shows up as a slope.
6. Ablations. Switch off one technique at a time to see which one carries the gain.

Start every trial from a clean environment, calibrate any LLM judge against human grades, and read the transcripts: they show whether a failure came from the context or from the model.

Worked example: a support agent, before and after

The scenario and numbers below are illustrative, built to show the arithmetic of a context budget, not measurements from a real system. An order-support agent is on its 25th model call of a long ticket about a damaged blender:

The "before" agent showed all four failure modes. It quoted a 14-day return window from an outdated article while its instructions said 30 days (clash), told the customer a failed refund had been issued and kept repeating it (poisoning), called update_shipping_address for a billing question (confusion), and fetched the same order four times (distraction). The "after" agent removes each at its source: superseded articles are filtered out at retrieval, the notes carry a correction line, the toolset is small enough to choose from, and a compaction trigger at 60,000 tokens stays as a safety net for long tickets.

To prove the change, run 30 tickets the old agent failed or escalated, three times per version. Grade outcomes against the order system and policy accuracy with a calibrated LLM judge, then compare success rate, pass^3, and tokens and cost per resolved ticket; if resolutions fall along with tokens, the ablations show which technique to roll back. Designing memory layers and an evaluation harness like this is the subject of the final modules of Context Engineering and Memory for AI Agents, which closes with a capstone: an end-to-end agent with persistent memory, evaluated for reliability.

A context engineering checklist

• You can print the exact context of any model call from your logs.
• The system prompt is minimal and sectioned, with nothing volatile at the top.
• A person could pick the right tool without hesitation; large catalogs sit behind tool search.
• Every tool returns only the fields the next decision needs, with pagination and a note when output is cut.
• Old tool results are cleared and history is compacted at a trigger you chose, with a summary prompt tuned for recall.
• The agent keeps structured notes and overwrites claims that turn out to be wrong.
• Sub-agents handle only independent, read-heavy work and return short summaries.
• An eval set of 20 to 50 real tasks tracks success, pass^k, and tokens and cost per successful task.

Frequently Asked Questions (FAQ)

What is context engineering in simple terms? Context engineering is deciding what an AI model sees each time it is called: its instructions, tool definitions, retrieved documents, memory, conversation history and tool results. The goal, in Anthropic's words, is "the smallest possible set of high-signal tokens" that makes the desired outcome likely. It matters most for agents, whose context changes on every step.

Who coined the term context engineering? No single coinage is documented. The term gained traction in June 2025: Walden Yan of Cognition called it "effectively the #1 job of engineers building AI agents" on 12 June, Shopify CEO Tobi Lütke wrote on 19 June that he preferred it to prompt engineering, LangChain's Harrison Chase defined it on 23 June and Andrej Karpathy endorsed it on 25 June. Anthropic's engineering guide followed on 29 September 2025.

Is context engineering replacing prompt engineering? No. Anthropic calls context engineering "the natural progression of prompt engineering," and Harrison Chase calls prompt engineering "a subset of context engineering." Clear instructions still matter, but they are only the part of the context you write by hand; code assembles the rest on every call.

What is context rot? Context rot is the decline in a model's accuracy and recall as the number of tokens in its context grows. Anthropic's documentation uses the term, and Chroma's July 2025 report, which tested 18 models, found significantly higher performance on focused prompts of about 300 tokens than on full inputs of about 113k tokens that also contained the relevant information.

Do bigger context windows make context engineering unnecessary? No. Current Claude models accept 1M tokens, yet long contexts still degrade, and cost and latency grow with context size. Anthropic expects that "context windows of all sizes will be subject to context pollution and information relevance concerns." A larger window raises the ceiling; it does not decide what deserves to be in it.

Conclusion

Context engineering treats everything a model reads as a budget to manage, not a bucket to fill. The term spread in June 2025, and the practice has since become concrete: shape tool output before it enters the context, retrieve just in time, clear and compact history with summaries tuned for recall, keep structured notes that record corrections, isolate independent work in sub-agents, and keep the prefix stable so caching keeps working. Each change can help or hurt, so pair them with an eval set built from real failures and compare success, consistency and cost per successful task before and after.