Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Agents go Astray: Course-Correcting SWE Agents with PRMs

Shubham Gandhi, Jason Tsay, Jatin Ganhotra, Kiran Kate, Yara Rizk

TL;DR

The paper tackles trajectory-level inefficiencies in large-language-model SWE agents by introducing SWE-PRM, an inference-time Process Reward Model that intermittently provides taxonomy-grounded, natural-language guidance to steer reasoning without modifying the base policy. By evaluating on the SWE-bench Verified benchmark with both open-weight and closed-source PRMs, the authors show that taxonomy-guided closed-source PRMs can raise overall resolution from $40.0\%$ to $50.6\%$, with the largest gains on medium and hard tasks, at an added cost of about $0.2$ per instance. The work systematically analyzes feedback variants and demonstrates that taxonomy-based guidance outperforms unguided or action-prescribing variants, offering a practical, scalable approach to improving SWE agent reliability and efficiency. The study discusses broader implications for process-aware guidance, enabling more efficient deployment of LLM agents in complex, long-horizon tasks, and outlines directions for cost-efficient invocations and cross-domain generalization.

Abstract

Large Language Model (LLM) agents are increasingly deployed for complex, multi-step software engineering (SWE) tasks. However, their trajectories often contain costly inefficiencies, such as redundant exploration, looping, and failure to terminate once a solution is reached. Prior work has largely treated these errors in a post-hoc manner, diagnosing failures only after execution. In this paper, we introduce SWE-PRM, an inference-time Process Reward Model (PRM) that intervenes during execution to detect and course-correct trajectory-level errors. Our PRM design leverages a taxonomy of common inefficiencies and delivers lightweight, interpretable feedback without modifying the underlying policy. On SWE-bench Verified, closed-source PRMs improve resolution from 40.0% to 50.6% (+10.6 p.p.), with the largest gains on medium and hard tasks. Among feedback strategies, taxonomy-guided PRMs outperform unguided or explicit action-prescriptive variants, increasing success rate while reducing trajectory length. These benefits come at an acceptable added inference cost of as low as $0.2, making PRMs a practical and scalable mechanism for improving SWE agents' reliability and efficiency.

When Agents go Astray: Course-Correcting SWE Agents with PRMs

TL;DR

The paper tackles trajectory-level inefficiencies in large-language-model SWE agents by introducing SWE-PRM, an inference-time Process Reward Model that intermittently provides taxonomy-grounded, natural-language guidance to steer reasoning without modifying the base policy. By evaluating on the SWE-bench Verified benchmark with both open-weight and closed-source PRMs, the authors show that taxonomy-guided closed-source PRMs can raise overall resolution from to , with the largest gains on medium and hard tasks, at an added cost of about per instance. The work systematically analyzes feedback variants and demonstrates that taxonomy-based guidance outperforms unguided or action-prescribing variants, offering a practical, scalable approach to improving SWE agent reliability and efficiency. The study discusses broader implications for process-aware guidance, enabling more efficient deployment of LLM agents in complex, long-horizon tasks, and outlines directions for cost-efficient invocations and cross-domain generalization.

Abstract

Large Language Model (LLM) agents are increasingly deployed for complex, multi-step software engineering (SWE) tasks. However, their trajectories often contain costly inefficiencies, such as redundant exploration, looping, and failure to terminate once a solution is reached. Prior work has largely treated these errors in a post-hoc manner, diagnosing failures only after execution. In this paper, we introduce SWE-PRM, an inference-time Process Reward Model (PRM) that intervenes during execution to detect and course-correct trajectory-level errors. Our PRM design leverages a taxonomy of common inefficiencies and delivers lightweight, interpretable feedback without modifying the underlying policy. On SWE-bench Verified, closed-source PRMs improve resolution from 40.0% to 50.6% (+10.6 p.p.), with the largest gains on medium and hard tasks. Among feedback strategies, taxonomy-guided PRMs outperform unguided or explicit action-prescriptive variants, increasing success rate while reducing trajectory length. These benefits come at an acceptable added inference cost of as low as $0.2, making PRMs a practical and scalable mechanism for improving SWE agents' reliability and efficiency.

Paper Structure

This paper contains 40 sections, 5 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Repository-Level Code Generation
  4. Improving LLM Agents
  5. Error analysis and taxonomies.
  6. Search-based improvements.
  7. Process optimization and recovery.
  8. Reward models for agent improvement.
  9. Methodology
  10. Task and Architecture
  11. PRM as Course-Corrector
  12. Motivation and Taxonomy
  13. Specification Errors (violations of task setup).
  14. Reasoning Errors (decision-making failures).
  15. Coordination Errors (multi-step process management failures).
  16. ...and 25 more sections

Figures (2)

  • Figure 1: SWE-PRM helps mitigate trajectory-level suboptimalities in SWE agents.
  • Figure 2: Difficulty-wise instances resolved out of 500 SWE-bench Verified instances (194 Easy, 261 Medium, 45 Hard). PRM$_D$ with Claude-Sonnet-4 yields the strongest gains across all tiers.