Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents

Zhaoxi Zhang, Yitong Duan, Yanzhi Zhang, Yiming Xu, Jiyan He, Yunfang Wu

TL;DR

The paper addresses repository-scale issue localization in large OSS under context-window limits. It introduces RepoNavigator, an LLM agent that uses a single execution-aware jump tool to trace symbol definitions, trained end-to-end with Group Reference Policy Optimization (GRPO). The approach achieves state-of-the-art localization performance across SWE benchmarks and model sizes, even surpassing some larger baselines and closed-source systems in training-free configurations. It argues that aligning tooling with execution structure and using a compact, capable tool yields robustness and scalability, with potential extensions to other languages and programming paradigms.

Abstract

Locating the files and functions requiring modification in large open-source software (OSS) repositories is challenging due to their scale and structural complexity. Existing large language model (LLM)-based methods typically treat this as a repository-level retrieval task and rely on multiple auxiliary tools, which overlook code execution logic and complicate model control. We propose RepoNavigator, an LLM agent equipped with a single execution-aware tool-jumping to the definition of an invoked symbol. This unified design reflects the actual flow of code execution while simplifying tool manipulation. RepoNavigator is trained end-to-end via Reinforcement Learning (RL) directly from a pretrained model, without any closed-source distillation. Experiments demonstrate that RL-trained RepoNavigator achieves state-of-the-art performance, with the 7B model outperforming 14B baselines, the 14B model surpassing 32B competitors, and even the 32B model exceeding closed-source models such as Claude-3.7. These results confirm that integrating a single, structurally grounded tool with RL training provides an efficient and scalable solution for repository-level issue localization.

One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents

TL;DR

The paper addresses repository-scale issue localization in large OSS under context-window limits. It introduces RepoNavigator, an LLM agent that uses a single execution-aware jump tool to trace symbol definitions, trained end-to-end with Group Reference Policy Optimization (GRPO). The approach achieves state-of-the-art localization performance across SWE benchmarks and model sizes, even surpassing some larger baselines and closed-source systems in training-free configurations. It argues that aligning tooling with execution structure and using a compact, capable tool yields robustness and scalability, with potential extensions to other languages and programming paradigms.

Abstract

Locating the files and functions requiring modification in large open-source software (OSS) repositories is challenging due to their scale and structural complexity. Existing large language model (LLM)-based methods typically treat this as a repository-level retrieval task and rely on multiple auxiliary tools, which overlook code execution logic and complicate model control. We propose RepoNavigator, an LLM agent equipped with a single execution-aware tool-jumping to the definition of an invoked symbol. This unified design reflects the actual flow of code execution while simplifying tool manipulation. RepoNavigator is trained end-to-end via Reinforcement Learning (RL) directly from a pretrained model, without any closed-source distillation. Experiments demonstrate that RL-trained RepoNavigator achieves state-of-the-art performance, with the 7B model outperforming 14B baselines, the 14B model surpassing 32B competitors, and even the 32B model exceeding closed-source models such as Claude-3.7. These results confirm that integrating a single, structurally grounded tool with RL training provides an efficient and scalable solution for repository-level issue localization.
Paper Structure (46 sections, 11 equations, 5 figures, 5 tables)

This paper contains 46 sections, 11 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Agentic Training
  4. Software Engineering Agents
  5. Method
  6. Problem Formulation
  7. Agent Architecture
  8. Jump: Symbol Resolution
  9. Syntactic Analysis
  10. Lexical Scope Resolution
  11. Static Type Inference
  12. Import Dependency Graph
  13. Reasoning--Action Loop
  14. Reinforcement Learning
  15. Experiment
  16. ...and 31 more sections

Figures (5)

  • Figure 1: Illustration of a LLM navigating through a code repository. The LLM is equipped with a single yet powerful tool: jump, which is realized through a language server.
  • Figure 2: Overview of our RepoNavigator. During the rollout phrase, the agent can call the jump tool, and the language server will return the definition code of the symbol. This process is trained by reinforcement learning.
  • Figure 3: Ablation study: comparison between RepoNavigator with training free, RFT, GRPO with pure outcome and hybrid reward on Qwen2.5-7B-Instruct.
  • Figure 4: Scaling law of tool-calling, where Pre and Post denote the corresponding metric before and after the RL training.
  • Figure 5: Venn graph illustrating access scope of jump. Compared with the repository scope, the access scope has a much higher IoU with the groundtruth set.