Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RGFL: Reasoning Guided Fault Localization for Automated Program Repair Using Large Language Models

Melika Sepidband, Hamed Taherkhani, Hung Viet Pham, Hadi Hemmati

TL;DR

The paper addresses the challenge of fault localization in large-scale software repositories for automated program repair by introducing RGFL, a reasoning-guided localization framework. RGFL inserts an explicit per-candidate reasoning step that generates bug-relevant explanations for files and code elements, then uses these explanations in a hierarchical, two-stage ranking to focus the repair model on the most promising regions. Across SWE-bench Verified, Lite, and Java datasets, RGFL improves file-level localization (e.g., Hit@1 and MRR) and element-level localization (Exact Match), and yields significant gains in end-to-end repair success when integrated with Agentless. A counterfactual ablation quantifies the contributions of file and element localization to repair performance, highlighting the pivotal role of accurate localization for practical, scalable APR in real-world codebases.

Abstract

Fault Localization (FL) is a critical step in Automated Program Repair (APR), and its importance has increased with the rise of Large Language Model (LLM)-based repair agents. In realistic project-level repair scenarios, software repositories often span millions of tokens, far exceeding current LLM context limits. Consequently, models must first identify a small, relevant subset of code, making accurate FL essential for effective repair. We present a novel project-level FL approach that improves both file- and element-level localization. Our method introduces a hierarchical reasoning module that (i) generates structured, bug-specific explanations for candidate files and elements, and (ii) leverages these explanations in a two-stage ranking scheme combining LLM-based and embedding-based signals. We further propose a counterfactual upper-bound analysis to quantify the contribution of each localization stage to repair success. We evaluate our approach on Python and Java projects from SWE-bench Verified, Lite, and Java. Compared to state-of-the-art baselines, including Agentless and OpenHands, our method consistently improves localization accuracy. On SWE-bench Verified, file-level Hit@1 improves from 71.4% to 85%, and MRR from 81.8% to 88.8%. At the element level, Exact Match under top-3 files increases from 36% to 69%. Integrating our localization into Agentless yields a 12.8% end-to-end repair success improvement.

RGFL: Reasoning Guided Fault Localization for Automated Program Repair Using Large Language Models

TL;DR

The paper addresses the challenge of fault localization in large-scale software repositories for automated program repair by introducing RGFL, a reasoning-guided localization framework. RGFL inserts an explicit per-candidate reasoning step that generates bug-relevant explanations for files and code elements, then uses these explanations in a hierarchical, two-stage ranking to focus the repair model on the most promising regions. Across SWE-bench Verified, Lite, and Java datasets, RGFL improves file-level localization (e.g., Hit@1 and MRR) and element-level localization (Exact Match), and yields significant gains in end-to-end repair success when integrated with Agentless. A counterfactual ablation quantifies the contributions of file and element localization to repair performance, highlighting the pivotal role of accurate localization for practical, scalable APR in real-world codebases.

Abstract

Fault Localization (FL) is a critical step in Automated Program Repair (APR), and its importance has increased with the rise of Large Language Model (LLM)-based repair agents. In realistic project-level repair scenarios, software repositories often span millions of tokens, far exceeding current LLM context limits. Consequently, models must first identify a small, relevant subset of code, making accurate FL essential for effective repair. We present a novel project-level FL approach that improves both file- and element-level localization. Our method introduces a hierarchical reasoning module that (i) generates structured, bug-specific explanations for candidate files and elements, and (ii) leverages these explanations in a two-stage ranking scheme combining LLM-based and embedding-based signals. We further propose a counterfactual upper-bound analysis to quantify the contribution of each localization stage to repair success. We evaluate our approach on Python and Java projects from SWE-bench Verified, Lite, and Java. Compared to state-of-the-art baselines, including Agentless and OpenHands, our method consistently improves localization accuracy. On SWE-bench Verified, file-level Hit@1 improves from 71.4% to 85%, and MRR from 81.8% to 88.8%. At the element level, Exact Match under top-3 files increases from 36% to 69%. Integrating our localization into Agentless yields a 12.8% end-to-end repair success improvement.
Paper Structure (28 sections, 5 figures, 6 tables)

This paper contains 28 sections, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Research Questions
  3. Contributions
  4. Background and Related Works
  5. Automated Program Repair (APR)
  6. Fault Localization Techniques and the Role of LLMs
  7. LLM-based APR Agents
  8. Agent-based end-to-end APR systems:
  9. Pipeline-based APR systems:
  10. LLM-based fault localization and issue localization methods:
  11. Positioning of our approach:
  12. Methodology
  13. Reasoning-based Localization Strategy
  14. Experiment Settings
  15. Dataset and Model
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Motivating example from SWE-bench Verified (Astropy library) showing that reasoning-guided element localization identifies the true fault-inducing method (HTML.write), while Agentless selects only superficially related elements.
  • Figure 2: Overview of our reasoning-based fault localization in APR
  • Figure 3: File localization performance (Hit@k, Recall@k) with and without LLM-based reasoning on the SWE-bench Verified dataset.
  • Figure 4: Cross-dataset comparison of file localization performance (Hit@k) across different methods.
  • Figure 5: The distribution of unresolved cases across localization stages. (file, element, and line level). There are in total 28 issues with wrong localized files, 111 (28+13+70) issues with wrong localized elements, and 175 (28+70+77) issues that have wrong localized lines, in our generated patches