DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal

TL;DR

DARS tackles suboptimal decision-making and compute inefficiency in LLM-powered coding agents by introducing Dynamic Action Re-Sampling, which branches the agent’s trajectory only at key decision points in a depth-first traversal to provide complete trajectory feedback before exploring alternatives. The method augments a base SWE agent with diff-based editing and new actions ( Execute Server, Execute IPython, Search Repo, Undo Edit ), and employs a branching and expansion strategy plus a reviewer-driven trajectory selection pipeline to generate and select high-quality patches. Empirical results on the SWE-Bench Lite benchmark demonstrate state-of-the-art open-source performance (pass@1 around 47%), with favorable cost-scaling and long-horizon planning benefits, validating DARS as a practical, scalable approach for automated software engineering tasks. The authors also release their data, models, and tooling to enable reproducible research and further improvement in coding agents.

Abstract

Large Language Models (LLMs) have revolutionized various domains, including natural language processing, data analysis, and software development, by enabling automation. In software engineering, LLM-powered coding agents have garnered significant attention due to their potential to automate complex development tasks, assist in debugging, and enhance productivity. However, existing approaches often struggle with sub-optimal decision-making, requiring either extensive manual intervention or inefficient compute scaling strategies. To improve coding agent performance, we present Dynamic Action Re-Sampling (DARS), a novel inference time compute scaling approach for coding agents, that is faster and more effective at recovering from sub-optimal decisions compared to baselines. While traditional agents either follow linear trajectories or rely on random sampling for scaling compute, our approach DARS works by branching out a trajectory at certain key decision points by taking an alternative action given the history of the trajectory and execution feedback of the previous attempt from that point. We evaluate our approach on SWE-Bench Lite benchmark, demonstrating that this scaling strategy achieves a pass@k score of 55% with Claude 3.5 Sonnet V2. Our framework achieves a pass@1 rate of 47%, outperforming state-of-the-art (SOTA) open-source frameworks.

Figures (25)

• Figure 1: Overview of our DARS scaling method. DARS processes issue-related information and generates multiple patches using the Expansion mechanism. These patches are then evaluated by our Reviewer LLM, which assigns scores based on predefined Score Rubrics, ultimately selecting the best patch for output.
• Figure 2: This figure presents coverage variation vs k. Here $\infty$ corresponds to submission of all the patches generated for an issue.
• Figure 3: Repo-wise coverage of for each model
• Figure 4: Variation of coverage with maximum branch depth
• Figure 5: Variation of coverage vs number of iterations