Agent Context Window Management: Keep the Evidence

Agent context window management is the discipline of keeping the next decision supplied with the right evidence, not keeping the conversation short. Compact the conversation without compacting away the contract: the current goal, constraints, unresolved questions, action state, and references needed to verify what happens next must survive. Everything else can compete for space.

A longer window does not remove this responsibility. A shorter summary does not prove it has been met. The useful question is whether the resumed agent can make the required decision from recoverable, current evidence. This is a design guide, not a report of measured compaction gains.

Separate the context window from durable task state

Treat the prompt as a view over state, not the only place state exists. Conversation history mixes requests, abandoned plans, observations, copied documents, and executed actions. Those categories need different retention rules.

The distinction builds on context engineering as working-set selection. Selection happens before every inference, not only when the provider rejects an oversized request.

Material	Keep in the active window	Keep outside the window
Task contract	Current objective, acceptance criteria, applicable constraints	Versioned request and change history
Authority	Current permitted scope and relevant restrictions	Authoritative policy and approval records
Progress	Verified results, pending work, unknown outcomes	Operation ledger and verifier artifacts
Evidence	Relevant excerpts, source identity, unresolved conflicts	Addressable source snapshots and retrieval metadata
Exploration	Useful findings and why a path was rejected	Recoverable search results and diagnostic logs

External storage is not permission to retain everything forever. Apply the same access scope, retention limits, and deletion rules as the underlying task. A context manager should not create an unrestricted second copy of sensitive data.

The model may propose progress notes, but those notes should not overwrite authoritative execution state. “The update probably worked” belongs under uncertainty until an independent check resolves it. It must not become “update completed” merely because a summarizer prefers concise sentences.

Budget for the next turn, not the current transcript

Count the assembled request: instructions, tool schemas, selected messages, retrieved evidence, and any protocol overhead exposed by the provider. Then reserve room under that provider’s accounting rules for generation and the next bounded tool response. Models and APIs differ in how they account for output and reasoning; there is no universal token threshold to copy into every harness.

A workable admission rule is: do not start a step whose expected request and response cannot fit within the configured limits with a safety margin. Estimate tool-result size before dispatch, enforce result limits, and check the assembled request again before the next model call. Byte limits help contain output but are not exact token counts.

Use two kinds of compaction trigger:

• Capacity triggers: the next request approaches the input budget, an unusually large result arrives, or a tool schema expansion consumes the reserve.
• Task-boundary triggers: a research branch closes, an implementation phase finishes, or verified intermediate work can be represented by a compact state record.

A phase boundary is often easier to summarize coherently than an arbitrary point inside a tool exchange. Preserve provider-required message structure, tool-call identifiers, and result pairing. Do not delete half of a call/result exchange and expect the next request to remain valid.

If mandatory instructions and minimum evidence already exceed the budget, summarization is not an adequate escape hatch. Reduce the scope of the next decision, retrieve narrower evidence, or use a suitable supported model. Never silently drop a required constraint to make the request fit.

Make compaction preserve an evidence contract

Anthropic’s context-engineering guidance describes compaction, structured notes, and just-in-time retrieval as techniques for long-running agents. It also warns that aggressive compaction can remove details whose importance appears later. Turn that warning into fields the harness can inspect.

A useful compacted record answers six questions:

1. What is the current task, including later corrections?
2. Which constraints and permission limits still apply?
3. What has been verified, and where is the evidence?
4. What remains unknown, contradictory, or blocked?
5. Which actions are pending, in progress, or of uncertain outcome?
6. What should be retrieved before the next decision?

Keep exact identifiers and material constraints out of free-form paraphrase where possible. A ticket number, revision, allowed destination, or required date should survive as a typed value or exact excerpt. Descriptive prose can explain the value; it should not be the only copy.

Consider an illustrative document-release task. The user requested a draft for review, a tool created an artifact, and a later message changed the target audience. A bad summary says “report ready; send it.” A useful record preserves “draft only,” the updated audience, the artifact reference, the checks already run, and the fact that distribution is not authorized. Compression must not convert preparation into permission.

Validate the proposed record before replacing the working context. Required fields must be present; cited artifacts must resolve; preserved action states must agree with the ledger; explicit constraints must not disappear. These checks catch structural losses. They do not establish semantic completeness, so retain access to the source history for bounded follow-up reads.

Rehydrate evidence before it becomes a decision

A reference is useful only if it can recover the right evidence. A path to a mutable file is not the same thing as a reference to the version the agent actually read. Store an artifact revision or digest when available, plus a selector such as a section, record ID, or line span. Include enough description to explain why the source matters without forcing a full reread.

Rehydration should be a narrow, explicit operation:

1. Identify the claim or pending decision that lacks sufficient support.
2. Resolve its source reference within the current task’s access scope.
3. Read the smallest complete passage or record needed, including relevant qualifications.
4. Check whether the task needs historical evidence or current state.
5. Attach the evidence and its freshness to the next decision.

The fourth step matters. A stored snapshot can prove what a tool returned earlier, while a fresh read establishes whether a mutable condition still holds. Neither substitutes for the other. Before changing a record, for example, retrieve its current revision and enforce the write precondition at execution time; a fresh read alone does not prevent a later race.

If a source has expired, disappeared, or become inaccessible, preserve that limitation. Retrieve an authorized replacement if one exists, narrow the conclusion, or block the dependent action. Do not let the existence of a fluent summary conceal the absence of its evidence.

Recovery also needs a budget. Repeatedly loading the same large document can erase compaction savings and create a read-summarize-read loop. Record which evidence is missing, cache scoped excerpts where appropriate, and escalate a persistent retrieval failure rather than repeating it indefinitely.

Preserve trust boundaries through every summary

Retrieved content remains data after summarization. An instruction embedded in a document must not become a top-level operating rule because the summarizer removed quotation marks or provenance. Keep source trust, applicable scope, and instruction hierarchy explicit when constructing the next request.

The same applies to negative evidence. “No matching record in the first page” is not “no matching record.” Retain truncation flags, pagination state, failed reads, and coverage limits whenever they affect a conclusion. A compacted answer should not sound more certain than the evidence that produced it.

Keep operational truth separate from model recollection. The recovery-checkpoint design covers durable intent, uncertain effects, and restart reconciliation. Context compaction should reference that state, not recreate it from a narrative. A timed-out write remains of unknown outcome until reconciliation establishes what happened; removing its raw tool output does not reset it to “not attempted.”

This boundary also prevents stale notes from granting authority. Load applicable policy from its authoritative source and revalidate action scope when required. A memory of earlier permission is evidence to inspect, not a mechanism for bypassing changed restrictions.

Evaluate compaction by what the agent can still do

The paper Lost in the Middle found position-sensitive performance on the models and multi-document question-answering and key-value retrieval tasks it studied. That supports testing evidence placement. It does not establish one universal decay curve for current models or your workload.

Evaluate the actual harness at controlled compaction boundaries. Hold the task, source evidence, permissions, verifier, and retry policy fixed. Compare a supported full-history baseline with candidate retention policies, including repeated compactions over the same task. Keep boundary-selection rules explicit so a policy is not evaluated only at convenient moments.

Include cases that require more than recalling a prominent fact:

• A later correction supersedes an earlier requirement.
• A small exception changes which action is permitted.
• Two sources disagree and neither can simply be discarded.
• A source changes after its snapshot was summarized.
• A tool result is incomplete or unavailable.
• A pending action has an unknown outcome when compaction occurs.
• Several rounds of summarization threaten to erase uncertainty.

These are proposed test cases, not reported passes. Use independent outcome checks, and inspect forbidden effects as well as final answers. A shorter prompt that produces an unsupported decision is a failed candidate even if its prose is convincing.

Record verified completion, lost constraints, unsupported claims, retrieval failures, duplicate effects, total tokens, elapsed time, and human recovery. Measure the entire task, including compaction and rehydration calls. The agent cost-optimization method applies directly: hold the outcome gate fixed before claiming that fewer input tokens made the system cheaper.

Keep the contract small, and the evidence reachable

Start with one explicit context-assembly policy: preserve the current task contract, retain verified progress and uncertainty, bound incoming results, and move recoverable bulk behind scoped references. Compact at understandable boundaries, validate the record, and retrieve evidence when the next decision needs it.

The success criterion is not how much text disappeared. It is whether the agent still knows what it is allowed to do, what remains unfinished, and how to verify its next step.