We ran a single agent session through all 89 sequential tasks of Terminal Bench 2.0 or over 80 million tokens, with no measurable accuracy loss versus running each task in its own fresh session. We didn't use compaction.
Compaction is the standard fix for finite context windows, but it has problems. The obvious one is context loss - you can't compress 300,000 tokens of work into a sub-20,000 summary. Only the most salient information survives, and a single model unanimously decides what's worth keeping (which also makes it prone to hallucination and bias). There are other issues, like the whole process halting for a while, but those don't hurt agent quality as much.
Out solution is that the agent annotates its work as it goes, and we use that to progressively evict only the information that's no longer relevant. We split work into two types, exploration and action. Exploration gathers the information needed to reach a goal; action acts on what was collected earlier. A single piece of information can be used by several independent actions, so it's important to track which actions depend on what. We do this with a single tool the agent uses to mark when it starts and finishes exploration or action work, and what each action depends on. This organizes the context window into a series of chunks, where action chunks depend on one or more exploration chunks - a graph we can use to evict orphaned chunks.
Eviction runs in a set order once the session hits an arbitrary token limit. First it removes completed action chunks; if still over, it moves to exploration chunks with no dependent actions, re-adjusting as the agent annotates new work. Eviction is also progressive inside each chunk, since not all tool calls carry equal information. Order of eviction is the following:
We forked the Pi.dev CLI and benchmarked it against Terminal Bench 2.0, SWE-bench Lite, Recovery Bench, and LongCLI Bench. Across all four, our approach and the isolated-session baseline differ by at most 3 points in either direction - within run-to-run variance.
Compaction is the standard fix for finite context windows, but it has problems. The obvious one is context loss - you can't compress 300,000 tokens of work into a sub-20,000 summary. Only the most salient information survives, and a single model unanimously decides what's worth keeping (which also makes it prone to hallucination and bias). There are other issues, like the whole process halting for a while, but those don't hurt agent quality as much.
Out solution is that the agent annotates its work as it goes, and we use that to progressively evict only the information that's no longer relevant. We split work into two types, exploration and action. Exploration gathers the information needed to reach a goal; action acts on what was collected earlier. A single piece of information can be used by several independent actions, so it's important to track which actions depend on what. We do this with a single tool the agent uses to mark when it starts and finishes exploration or action work, and what each action depends on. This organizes the context window into a series of chunks, where action chunks depend on one or more exploration chunks - a graph we can use to evict orphaned chunks.
Eviction runs in a set order once the session hits an arbitrary token limit. First it removes completed action chunks; if still over, it moves to exploration chunks with no dependent actions, re-adjusting as the agent annotates new work. Eviction is also progressive inside each chunk, since not all tool calls carry equal information. Order of eviction is the following:
1. chain-of-thought contents 2. search results, directory listings, grep/glob output 3. arbitrary bash command runs and their outputs 4. file reads
We forked the Pi.dev CLI and benchmarked it against Terminal Bench 2.0, SWE-bench Lite, Recovery Bench, and LongCLI Bench. Across all four, our approach and the isolated-session baseline differ by at most 3 points in either direction - within run-to-run variance.
Paper - https://arxiv.org/pdf/2606.11213 Repo (fork of pi.dev): https://github.com/Kiz8-Team/pi-cwl