Context as a Tool: Context Management for Long-Horizon SWE-Agents

TL;DR

This work tackles the bottleneck of long-horizon reasoning in SWE agents by reframing context management as a callable, learnable tool called CAT. It introduces a structured context workspace, a long-term memory condenser, and a trajectory-level supervision pipeline (CaT-Generator) to train SWE-Compressor, which proactively compresses and reuses history. Through SWE-Bench-Verified experiments, CAT demonstrates superior solved rates and stable context usage versus ReAct and static baselines, approaching or surpassing larger models under the same budgets. The approach offers scalable, task-aware memory management for real-world code bases, enabling more reliable long-run reasoning in software-engineering agents.

Abstract

Agents based on large language models have recently shown strong potential on real-world software engineering (SWE) tasks that require long-horizon interaction with repository-scale codebases. However, most existing agents rely on append-only context maintenance or passively triggered compression heuristics, which often lead to context explosion, semantic drift, and degraded reasoning in long-running interactions. We propose CAT, a new context management paradigm that elevates context maintenance to a callable tool integrated into the decision-making process of agents. CAT formalizes a structured context workspace consisting of stable task semantics, condensed long-term memory, and high-fidelity short-term interactions, and enables agents to proactively compress historical trajectories into actionable summaries at appropriate milestones. To support context management for SWE-agents, we propose a trajectory-level supervision framework, CAT-GENERATOR, based on an offline data construction pipeline that injects context-management actions into complete interaction trajectories. Using this framework, we train a context-aware model, SWE-Compressor. Experiments on SWE-Bench-Verified demonstrate that SWE-Compressor reaches a 57.6% solved rate and significantly outperforms ReAct-based agents and static compression baselines, while maintaining stable and scalable long-horizon reasoning under a bounded context budget.

Figures (6)

• Figure 1: Overview of Cat with a structured context workspace for long-horizon reasoning.
• Figure 2: Example of structured context condensation in Cat during long-horizon tasks.
• Figure 3: Overview of the data construction and training pipeline for Cat. The process includes SWE instance collection, base ReAct trajectory generation, condenser point identification, structured summary generation, tool-based context injection, and supervised fine-tuning with rejection sampling.
• Figure 4: Context token usage and trajectory survival of Cat over interaction rounds on SWE-Bench-Verified.
• Figure 5: Comparison of scalability and efficiency between Cat and ReAct on SWE-Bench. (a) Cat exhibits an upward trend in performance as the interaction budget increases to 500 rounds, whereas ReAct saturates and degrades after 60 rounds. (b) Cat maintains stable context usage (35k tokens) via condensation, while ReAct rapidly exhausts the context window.