Context Engineering for AI Agents: A Developer's Guide

Most agent failures aren't model failures. You swap in a smarter model, the eval scores go up, and the agent still forgets what tool it just called, contradicts a decision it made three turns ago, or drowns a simple task in forty thousand tokens of retrieved documentation it never needed. That's not a reasoning problem. It's a context problem — and no amount of prompt tweaking fixes it, because the prompt was never the thing that was broken.

Context engineering is the discipline that actually fixes it: deciding what goes into the model's context window, in what order, and when it gets evicted. It's not a rebrand of prompt engineering. A prompt is static text you write once. Context is a dynamic, per-turn assembly problem — system instructions, tool schemas, retrieved documents, conversation history, and scratch memory, all competing for a fixed token budget, all changing shape as the agent runs.

This guide skips the theory-only version of that story. Every technique below ships with code you can actually run: a token budget allocator, a retrieval pipeline that doesn't dump every match into context, a rolling summarizer, and a sub-agent isolation pattern. If you're building anything more than a single-turn chatbot — a coding agent, a research agent, anything with tools and memory — this is the stuff that determines whether it stays coherent past turn twenty.

In this guide, we'll take a look at:

1. What context engineering is, and why it's not prompt engineering with a new name
2. The four core techniques: write, select, compress, isolate
3. Building a token budget manager
4. Selecting context without drowning the model in retrieval
5. Compressing context with rolling summarization
6. Isolating context with sub-agents
7. The failure modes that show up when you skip all of the above
8. Tools worth knowing about
9. What I'd actually do on a real project

What Is Context Engineering, And Why Isn't Prompt Engineering Enough?#

Prompt engineering optimizes a single static string: the wording of your system prompt, a few-shot example, the phrasing of an instruction. It matters, but it assumes the rest of the context — tool outputs, retrieved chunks, prior turns — is someone else's problem.

It isn't. Once an agent runs more than one step, most of what's in its context window wasn't written by you at turn one. It's tool results, file contents, search hits, and its own prior outputs, accumulating turn over turn. A model with a 1M-token context window doesn't fix this — it just delays the moment where irrelevant tokens start crowding out the signal. Anthropic and OpenAI both publish research on "context rot": model accuracy on a fixed task degrades as irrelevant context grows, even when the context window has room to spare. Long context is not the same as good context.

Context engineering treats the input to the model as an engineered system with four operations you actively manage:

• Write — persist information outside the context window so you can retrieve it later instead of keeping it live the whole time
• Select — pull back only the subset of written or historical context relevant to the current step
• Compress — shrink context that has to stay in-window but doesn't need full fidelity
• Isolate — give a sub-task its own clean context instead of letting it inherit the parent's entire history

Each one maps to a concrete piece of code. Let's build them.

The Four Core Techniques#

1. Write: Persist State Outside the Window#

The simplest fix for a bloated context window is to stop keeping everything in it. Anything the agent doesn't need on the very next step — a completed sub-task's full output, a large file it read, an intermediate calculation — should be written to a scratchpad (a file, a key-value store, a database row) and referenced by a short pointer instead of kept inline.

import json
import time
from pathlib import Path

class ContextStore:
    """Durable scratch storage the agent writes to instead of the live context."""

    def __init__(self, path: str = ".agent_scratch"):
        self.path = Path(path)
        self.path.mkdir(exist_ok=True)

    def write(self, key: str, value: str) -> str:
        record = {"value": value, "written_at": time.time()}
        (self.path / f"{key}.json").write_text(json.dumps(record))
        return f"[stored:{key}, {len(value)} chars]"  # this is what stays in context

    def read(self, key: str) -> str:
        record = json.loads((self.path / f"{key}.json").read_text())
        return record["value"]

store = ContextStore()

# Instead of keeping a 6,000-token file dump in the conversation:
pointer = store.write("readme_dump", huge_file_contents)
# the agent's context only ever sees:
# "Read src/README.md -> [stored:readme_dump, 6104 chars]"

The agent's context now holds a 40-character pointer instead of a 6,000-token file. If a later step genuinely needs the content, it calls store.read("readme_dump") — a deliberate, visible retrieval, not a standing tax on every turn.

2. Select: Retrieve Only What the Current Step Needs#

Selection is the operation people usually mean when they say "RAG," but it applies to conversation history and tool results just as much as it applies to a document store. We'll build a full example in the next section — the key discipline is: never dump a full search result set into context. Rank it, cut it, and justify every chunk that makes it in.

3. Compress: Shrink What Has to Stay#

Some context has to remain in-window — recent conversation turns, the active task description — but doesn't need to stay at full resolution forever. Compression trades fidelity for space, usually by summarizing older turns once they age past the point where exact wording matters. Full example below.

4. Isolate: Give Sub-Tasks a Clean Window#

If a sub-task doesn't need the parent's entire history to do its job, don't give it that history. Spin it up with only the inputs it needs, run it in its own context window, and return a small structured result to the parent. This is the single highest-leverage technique for long-running agents, and it's the one most homegrown agent loops skip entirely. Full example below.

Building a Token Budget Manager#

Before you can select or compress intelligently, you need to know what you're spending. A token budget manager allocates a fixed context window across categories — system instructions, tool schemas, memory, retrieved context, conversation — and tells you when a category is over budget instead of letting it silently eat the window.

import tiktoken

encoder = tiktoken.get_encoding("cl100k_base")

def count_tokens(text: str) -> int:
    return len(encoder.encode(text))

class TokenBudget:
    def __init__(self, total_budget: int, allocations: dict[str, float]):
        """
        total_budget: total tokens available for the assembled context
        allocations: fraction of the budget per category, must sum to <= 1.0
        """
        assert sum(allocations.values()) <= 1.0
        self.total_budget = total_budget
        self.limits = {k: int(total_budget * v) for k, v in allocations.items()}
        self.used = {k: 0 for k in allocations}

    def fits(self, category: str, text: str) -> bool:
        tokens = count_tokens(text)
        return self.used[category] + tokens <= self.limits[category]

    def add(self, category: str, text: str) -> bool:
        if not self.fits(category, text):
            return False
        self.used[category] += count_tokens(text)
        return True

    def report(self) -> dict:
        return {
            k: f"{self.used[k]}/{self.limits[k]} tokens ({self.used[k]/self.limits[k]:.0%})"
            for k in self.limits
        }

budget = TokenBudget(
    total_budget=32_000,
    allocations={
        "system": 0.05,      # 1,600 tokens — instructions, persona
        "tools": 0.10,       # 3,200 tokens — tool schemas
        "memory": 0.15,      # 4,800 tokens — summarized history / long-term memory
        "retrieval": 0.30,   # 9,600 tokens — RAG chunks
        "conversation": 0.40 # 12,800 tokens — recent raw turns
    },
)

for chunk in retrieved_chunks:  # ranked highest-relevance first
    if not budget.add("retrieval", chunk.text):
        break  # budget exhausted, stop adding lower-ranked chunks

print(budget.report())
# {'system': '1450/1600 (91%)', 'tools': '2890/3200 (90%)',
#  'memory': '3100/4800 (65%)', 'retrieval': '9600/9600 (100%)',
#  'conversation': '8200/12800 (64%)'}

The point isn't the exact percentages — it's that every category has an explicit ceiling, and low-ranked context gets dropped before assembly instead of silently truncated by the API when the whole thing overflows.

Selecting Context: Retrieval Without Drowning the Model#

A naive RAG pipeline embeds a query, pulls the top-k chunks by cosine similarity, and pastes all of them into context. That's how you end up with fifteen near-duplicate chunks about the same function and none about the edge case that actually matters. A better selection step reranks, deduplicates, and enforces a hard cap — driven by the token budget above, not an arbitrary "top 5."

import numpy as np

def cosine_sim(a: np.ndarray, b: np.ndarray) -> float:
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

def select_context(query_embedding: np.ndarray, candidates: list[dict],
                    budget: TokenBudget, similarity_floor: float = 0.72,
                    max_per_source: int = 2) -> list[dict]:
    """
    candidates: [{"text": str, "embedding": np.ndarray, "source": str}, ...]
    Ranks by similarity, drops near-duplicates and low-relevance hits,
    caps how many chunks can come from a single source, and stops
    once the retrieval budget is spent.
    """
    scored = [
        {**c, "score": cosine_sim(query_embedding, c["embedding"])}
        for c in candidates
    ]
    scored = [c for c in scored if c["score"] >= similarity_floor]
    scored.sort(key=lambda c: c["score"], reverse=True)

    selected, per_source_count = [], {}
    for c in scored:
        src = c["source"]
        if per_source_count.get(src, 0) >= max_per_source:
            continue  # already have enough from this file/doc
        if not budget.fits("retrieval", c["text"]):
            continue  # would blow the budget, skip and keep looking
        budget.add("retrieval", c["text"])
        selected.append(c)
        per_source_count[src] = per_source_count.get(src, 0) + 1

    return selected

Three things are doing the actual work here, and none of them are "call the vector DB":

• similarity_floor throws out results that matched on vocabulary overlap, not meaning — a 0.4-similarity hit is noise, not signal
• max_per_source stops one long file from monopolizing the retrieval budget with five overlapping chunks
• Budget-aware selection means the cutoff is "we're out of tokens," not an arbitrary top-k that either wastes budget or truncates mid-sentence

Compressing Context: Rolling Summarization#

Conversation history is the category that grows the fastest and needs to stay roughly in-window the longest. The fix isn't to summarize everything after every turn — that's expensive and lossy. It's to keep a sliding window of recent turns at full fidelity and periodically fold everything older into a running summary.

from anthropic import Anthropic

client = Anthropic()

class RollingMemory:
    def __init__(self, keep_recent: int = 6, summary_trigger: int = 12):
        self.turns: list[dict] = []
        self.summary: str = ""
        self.keep_recent = keep_recent
        self.summary_trigger = summary_trigger

    def add_turn(self, role: str, content: str):
        self.turns.append({"role": role, "content": content})
        if len(self.turns) >= self.summary_trigger:
            self._compress()

    def _compress(self):
        to_fold = self.turns[: -self.keep_recent]
        self.turns = self.turns[-self.keep_recent :]

        transcript = "
".join(f"{t['role']}: {t['content']}" for t in to_fold)
        response = client.messages.create(
            model="claude-sonnet-5",
            max_tokens=400,
            messages=[{
                "role": "user",
                "content": (
                    "Fold this conversation excerpt into the running summary below. "
                    "Preserve decisions made, facts established, and open questions. "
                    "Drop small talk and superseded information.

"
                    f"Running summary so far:
{self.summary or '(none yet)'}

"
                    f"New excerpt to fold in:
{transcript}"
                ),
            }],
        )
        self.summary = response.content[0].text

    def context_block(self) -> str:
        recent = "
".join(f"{t['role']}: {t['content']}" for t in self.turns)
        return f"Summary of earlier conversation:
{self.summary}

Recent turns:
{recent}"

The important design decision is keep_recent vs summary_trigger: compression only fires once there's enough slack (12 turns) to fold a meaningful batch (6 turns) into the summary, and it always leaves the most recent turns at full fidelity. Summarizing every single turn as it arrives produces a lossy, jittery summary and burns an LLM call per turn for no benefit.

Isolating Context: Sub-Agents With Clean Windows#

This is the technique that matters most for anything beyond a linear task, and the one that's easiest to skip because it feels like extra plumbing. If your agent has to research three unrelated things before it can act, don't do all three research steps in the main conversation — each one pollutes the shared context with tool calls and results the other two don't need. Spin up an isolated sub-agent per sub-task instead.

from anthropic import Anthropic

client = Anthropic()

def run_subagent(task: str, inputs: dict, tools: list[dict] | None = None) -> str:
    """
    Runs a task in a fresh context window — no parent conversation history,
    no unrelated tool results. Only `task` and `inputs` go in; only a short
    structured result comes back out.
    """
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        tools=tools or [],
        messages=[{
            "role": "user",
            "content": (
                f"Task: {task}

"
                f"Inputs: {inputs}

"
                "Return only the final result, no explanation of your process."
            ),
        }],
    )
    return response.content[0].text

def research_and_plan(feature_request: str) -> str:
    # Each sub-agent gets ONLY what it needs — none of them see each other's
    # tool calls, search results, or reasoning trace.
    api_findings = run_subagent(
        "Summarize how the existing auth module handles session tokens",
        {"feature_request": feature_request},
    )
    db_findings = run_subagent(
        "List the schema changes needed to support this feature",
        {"feature_request": feature_request},
    )
    risk_findings = run_subagent(
        "Identify breaking changes this feature could introduce",
        {"feature_request": feature_request},
    )

    # The PARENT context only ever holds three short summaries,
    # not three sub-agents' worth of tool calls and intermediate noise.
    return run_subagent(
        "Write an implementation plan from these findings",
        {
            "api": api_findings,
            "database": db_findings,
            "risks": risk_findings,
        },
    )

The parent's context grows by three short strings, not by three research trajectories' worth of tool calls. This is also why multi-agent frameworks that spawn sub-agents (Claude's own sub-agent support, LangGraph's subgraphs, CrewAI's crews) exist in the first place — isolation is the mechanism, "multi-agent" is just the name for using it deliberately.

Common Failure Modes#

Failure mode	What it looks like	Fix
Context rot	Accuracy degrades as irrelevant tokens accumulate, even with room left in the window	Compress and evict aggressively; don't rely on window size as a substitute for curation
Lost in the middle	The model attends well to the start and end of context, poorly to the middle	Put the highest-priority instructions and the most relevant retrieved chunk near the end, right before the query
Context poisoning	A hallucinated fact or wrong tool result gets referenced as true in later turns because it's sitting in context	Isolate exploratory/uncertain steps in sub-agents; only promote verified results to the parent context
Context distraction	Too many available tools or examples pull the model toward a plausible-but-wrong action	Select tools per-step instead of exposing the full tool registry on every call
Silent truncation	The context window overflows and the API (or your framework) truncates the oldest turns mid-conversation	Track usage with an explicit budget (like TokenBudget above) and compress before you hit the limit, not after

Tools Worth Knowing About#

You can build everything above from scratch — it's a few hundred lines of code, as shown. But a few tools handle pieces of it if you don't want to own them:

• MCP (Model Context Protocol) — standardizes how tools and resources get exposed to an agent, so tool schemas are loaded on demand instead of dumped into every system prompt (worth a read if you haven't set one up: our MCP server guide covers the basics)
• LangGraph — first-class support for sub-graphs, which is the isolation pattern above with a scheduler on top
• LlamaIndex — retrieval pipelines with built-in reranking and dedup, closer to production-ready than the select_context example above
• Vector databases (pgvector, Qdrant, Pinecone) — the "write" half of the write/select loop; pick based on whether you need it embedded in Postgres or standalone

What I'd Actually Do#

Start with the budget manager, even before you touch retrieval or memory. It's 40 lines of code and it turns "the agent's context is a mess" from a vibe into a set of numbers you can look at. Once you can see that tool schemas are eating 40% of your window, the fix is usually obvious — you're loading 30 tools when the current step needs 3.

After that, isolation gives you the best return for the least code. A single run_subagent function, used consistently for anything that doesn't need the full parent history, prevents most of the context rot and poisoning problems before they start — no summarization or reranking required. Add compression once conversations are genuinely running long, and treat retrieval quality (the select_context filters) as the last thing you tune, because a good compression and isolation setup means retrieval has far less noise to filter out in the first place.

Final Thoughts#

None of this is exotic — write, select, compress, isolate are operations you'd apply to any system that has to manage a scarce, ordered resource, because that's what a context window is. The mistake is treating context as something that just accumulates until the model gets smart enough to handle it. It won't. The window gets bigger, the noise grows with it, and the agents that stay coherent past turn fifty are the ones where someone decided, deliberately, what earns a place in context and what gets written down, summarized, or handled somewhere else.