Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Monte Carlo Tree Search for Execution-Guided Program Repair with Large Language Models

Yixuan Liang

TL;DR

The paper tackles the challenge of repository-scale automated program repair using large language models, which struggle with long-horizon reasoning and greedy decoding. It proposes CodePilot, a hybrid framework that combines Monte Carlo Tree Search with a Qwen3 backbone to perform execution-guided patch generation, anchored by hierarchical fault localization and confidence-calibrated refinement. Key contributions include a three-level localization pipeline, MCTS-guided patch synthesis, execution-driven self-refinement, task-specific fine-tuning with LoRA and PPO, and a robust data-preprocessing and evaluation setup. Empirical results on SWE-bench Lite show a 24.67% resolve rate with open-weight models, surpassing baselines and ablations confirming the value of MCTS, thinking mode reasoning, and iterative refinement. The approach demonstrates that integrating symbolic search with neural models can enable scalable, execution-aware software engineering automation and paves the way for more complex, multi-file repair capabilities.

Abstract

Automated program repair with large language models remains challenging at the repository level due to long-horizon reasoning requirements and the limitations of autoregressive decoding. We present CodePilot, a hybrid framework that integrates Monte Carlo Tree Search (MCTS) with large language models to enable execution-guided program repair for real-world GitHub issues. CodePilot performs hierarchical fault localization from repository to file and function level, explores diverse patch trajectories using MCTS, and leverages execution feedback as a reward signal to guide search and refinement. The framework further incorporates confidence-calibrated generation to selectively refine low-confidence outputs. Experiments on SWE-bench Lite demonstrate that CodePilot achieves a 24.67% issue resolution rate using open-weight models, outperforming comparable baselines. These results suggest that combining symbolic search with neural language models is an effective strategy for scalable, execution-aware software engineering automation.

Monte Carlo Tree Search for Execution-Guided Program Repair with Large Language Models

TL;DR

The paper tackles the challenge of repository-scale automated program repair using large language models, which struggle with long-horizon reasoning and greedy decoding. It proposes CodePilot, a hybrid framework that combines Monte Carlo Tree Search with a Qwen3 backbone to perform execution-guided patch generation, anchored by hierarchical fault localization and confidence-calibrated refinement. Key contributions include a three-level localization pipeline, MCTS-guided patch synthesis, execution-driven self-refinement, task-specific fine-tuning with LoRA and PPO, and a robust data-preprocessing and evaluation setup. Empirical results on SWE-bench Lite show a 24.67% resolve rate with open-weight models, surpassing baselines and ablations confirming the value of MCTS, thinking mode reasoning, and iterative refinement. The approach demonstrates that integrating symbolic search with neural models can enable scalable, execution-aware software engineering automation and paves the way for more complex, multi-file repair capabilities.

Abstract

Automated program repair with large language models remains challenging at the repository level due to long-horizon reasoning requirements and the limitations of autoregressive decoding. We present CodePilot, a hybrid framework that integrates Monte Carlo Tree Search (MCTS) with large language models to enable execution-guided program repair for real-world GitHub issues. CodePilot performs hierarchical fault localization from repository to file and function level, explores diverse patch trajectories using MCTS, and leverages execution feedback as a reward signal to guide search and refinement. The framework further incorporates confidence-calibrated generation to selectively refine low-confidence outputs. Experiments on SWE-bench Lite demonstrate that CodePilot achieves a 24.67% issue resolution rate using open-weight models, outperforming comparable baselines. These results suggest that combining symbolic search with neural language models is an effective strategy for scalable, execution-aware software engineering automation.
Paper Structure (20 sections, 28 equations, 4 figures, 2 tables)

This paper contains 20 sections, 28 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Foundation Model: Qwen3 Architecture
  5. Hierarchical Fault Localization
  6. MCTS for Patch Synthesis
  7. Execution-Driven Self-Refinement
  8. Confidence-Calibrated Generation
  9. Fine-Tuning Strategy
  10. Supervised Fine-tuning with LoRA
  11. Reinforcement Learning from Execution Feedback
  12. Data Preprocessing
  13. Repository Parsing and Code Chunking
  14. Issue Text Normalization
  15. Evaluation Metrics
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Overview of the CodePilot framework. The system takes a GitHub issue and repository as input, performs hierarchical fault localization to identify suspicious code regions, generates candidate patches via MCTS-guided synthesis, and iteratively refines solutions using execution feedback. Qwen3’s dual-mode reasoning enables both deep analysis and rapid generation.
  • Figure 2: Three-level hierarchical fault localization pipeline. The system progressively narrows down from the full repository to specific edit locations through file-level hybrid retrieval (combining dense embeddings and BM25) and function-level AST analysis with dependency ordering.
  • Figure 3: Illustration of MCTS-guided patch synthesis. The search tree explores diverse solution trajectories where each node represents a partial patch state. The four phases---Selection, Expansion, Simulation, and Backpropagation---iteratively refine the search toward high-reward patches validated by test execution.
  • Figure 4: Model indicator change chart.