OrcaLoca: An LLM Agent Framework for Software Issue Localization

TL;DR

OrcaLoca tackles the challenging problem of precise software issue localization in autonomous software engineering by integrating three novel mechanisms into a CodeGraph-driven LLM agent: priority-based action scheduling, action decomposition with relevance scoring, and distance-aware context pruning. The framework achieves open-source state-of-the-art function matching on SWE-bench Lite (65.33%) and improves final patch resolution by 6.33 percentage points when combined with patch-generation components. Key contributions include a dynamic ASQ for robust action planning, a multi-agent decomposition workflow to balance conciseness and completeness, and a context manager that prunes search data using graph-distance to bug locations. Together, these components yield more accurate localization, more efficient exploration, and a modular platform that can guide future ASE systems in integrating LLMs with precise code-search and repair workflows.

Abstract

Recent developments in Large Language Model (LLM) agents are revolutionizing Autonomous Software Engineering (ASE), enabling automated coding, problem fixes, and feature improvements. However, localization -- precisely identifying software problems by navigating to relevant code sections -- remains a significant challenge. Current approaches often yield suboptimal results due to a lack of effective integration between LLM agents and precise code search mechanisms. This paper introduces OrcaLoca, an LLM agent framework that improves accuracy for software issue localization by integrating priority-based scheduling for LLM-guided action, action decomposition with relevance scoring, and distance-aware context pruning. Experimental results demonstrate that OrcaLoca becomes the new open-source state-of-the-art (SOTA) in function match rate (65.33%) on SWE-bench Lite. It also improves the final resolved rate of an open-source framework by 6.33 percentage points through its patch generation integration.

Figures (5)

• Figure 1: Distribution and average of file / function match rate and resolved rate on SWE-Bench Lite LeaderBoard.
• Figure 2: An overview of OrcaLoca using a demonstrating example from issue django_14580. (a) shows an abbreviated version of the issue's problem statement, where the user emphasizes CreateModel and MigrationWriter. (b) presents the exploration sequence of our agent over a part of the whole CodeGraph. (c) provides details of the Action Scheduler Queue (ASQ). Specifically, action decomposition is applied from to and from to , as discussed in Section \ref{['score ranking']}. Additionally, techniques described in Section \ref{['scheduler queue']} are used to handle steps from to and to . (d) illustrates the distance-aware context pruning process, elaborated in Section \ref{['context manager']}. Finally, (e) shows the agent's final output. Please note this is a demonstration, experiments may use different configuration.
• Figure 3: Detailed examples for OrcaLoca solving redundancy and disambiguation problem.
• Figure 4: Unique localizations and solutions of open source agents.
• Figure 5: Internal structure of reproducer agent. \ref{['reproducer_0']} contents are labeled in blue, \ref{['reproducer_1']} in red and \ref{['reproducer_2']} in purple.