Lingxi: Repository-Level Issue Resolution Framework Enhanced by Procedural Knowledge Guided Scaling

TL;DR

Lingxi tackles the repository-level issue resolution problem by addressing two core limitations of prior LLM-based approaches: the lack of transferable procedural knowledge and a reliance on brute-force exploration. It constructs transferable procedural knowledge offline via a hierarchical abstraction workflow that extracts problem-solving reasoning from historical issue reports and patches, then applies knowledge-guided, parallel analysis to new issues. A dedicated analysis-synthesis pipeline integrates multiple knowledge sources to produce a unified diagnostic and plan, which is then executed by a planning and fixing duo of agents. On SWE-bench Verified, Lingxi achieves a Pass@1 of $74.6\%$, outperforming five state-of-the-art baselines by $5.4\%$ to $14.9\%$, with ablations showing that transferable knowledge is the key driver of gains and that scaling is effective only when guided by high-quality knowledge. The work demonstrates that extracting and fusing procedural knowledge from historical data can significantly enhance automated repair, with design patterns and coding practices emerging as the most impactful knowledge aspects across analysis, planning, and fixing.

Abstract

Driven by the advancements of Large Language Models (LLMs), LLM-powered agents are making significant improvements in software engineering tasks, yet struggle with complex, repository-level issue resolution. Existing agent-based methods have two key limitations. First, they lack of procedural knowledge (i.e., how an issue is fixed step-by-step and rationales behind it) to learn and leverage for issue resolution. Second, they rely on massive computational power to blindly explore the solution space. % To address those limitations, we propose Lingxi, an issue resolution framework that leverages procedural knowledge extracted from historical issue-fixing data to guide agents in solving repository-level issues. \ourTool first constructs this knowledge offline through a hierarchical abstraction mechanism, enabling agents to learn the how and why behind a fix, not just the final solution. During online application, it employs a knowledge-driven scaling method that leverages the procedural knowledge of similar issues to intelligently analyze the target issue from multiple perspectives, in sharp contrast to undirected, brute-force exploration. % Lingxi successfully resolves 74.6\% of bugs on the SWE-bench Verified benchmark in Past@1 setting, outperforming five state-of-the-art techniques by a significant margin (5.4\% to 14.9\%). Our comprehensive ablation study confirmed that the success of Lingxi comes directly from its use of procedural knowledge. Without it, the performance gains from scaling alone is negligible. Our qualitative study further shows that the ``design patterns $\&$ coding practices'' is the most critical knowledge aspect, and that the roles of different knowledge aspects switch across different stages (i.e., analysis, planning, and fixing).

Figures (5)

• Figure 1: Target Django issue D-32296 and its similar historical issues (D-30418 and D-23685)
• Figure 2: Example scenarios of an agent solving Django issue D-32296 1) with only tools, 2) with tools and raw data from historical issues, 3) with tools and transferable knowledge from historical issues
• Figure 3: Workflow of Lingxi.
• Figure 4: The prompt template for knowledge abstraction. Note that for saving space, we only keep the key component and use $\dots$ to denote the saving sentences.
• Figure 5: The Contribution of Transferable Procedural Knowledge within the Lingxi