Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finding the Needle in the Crash Stack: Industrial-Scale Crash Root Cause Localization with AutoCrashFL

Sungmin Kang, Sumi Yun, Jingun Hong, Shin Yoo, Gabin An

TL;DR

AutoCrashFL introduces an industrial-scale, crash-focused fault-localization agent built on an adapted AutoFL framework. It operates directly on crashdumps and a code repository via a clangd-powered interface, requiring no test coverage data, and aggregates evidence through multiple runs to produce concise, high-accuracy file rankings. On 454 SAP HANA crashes, AutoCrashFL achieves superior top-1 localization and robust performance across crash types, with practical insights into confidence calibration and tool usage patterns. The work demonstrates the viability of LLM-based debugging agents in production environments while outlining key limitations and avenues for improvement in explanation reliability and deep-code navigation.

Abstract

Fault Localization (FL) aims to identify root causes of program failures. FL typically targets failures observed from test executions, and as such, often involves dynamic analyses to improve accuracy, such as coverage profiling or mutation testing. However, for large industrial software, measuring coverage for every execution is prohibitively expensive, making the use of such techniques difficult. To address these issues and apply FL in an industrial setting, this paper proposes AutoCrashFL, an LLM agent for the localization of crashes that only requires the crashdump from the Program Under Test (PUT) and access to the repository of the corresponding source code. We evaluate AutoCrashFL against real-world crashes of SAP HANA, an industrial software project consisting of more than 35 million lines of code. Experiments reveal that AutoCrashFL is more effective in localization, as it identified 30% crashes at the top, compared to 17% achieved by the baseline. Through thorough analysis, we find that AutoCrashFL has attractive practical properties: it is relatively more effective for complex bugs, and it can indicate confidence in its results. Overall, these results show the practicality of LLM agent deployment on an industrial scale.

Finding the Needle in the Crash Stack: Industrial-Scale Crash Root Cause Localization with AutoCrashFL

TL;DR

AutoCrashFL introduces an industrial-scale, crash-focused fault-localization agent built on an adapted AutoFL framework. It operates directly on crashdumps and a code repository via a clangd-powered interface, requiring no test coverage data, and aggregates evidence through multiple runs to produce concise, high-accuracy file rankings. On 454 SAP HANA crashes, AutoCrashFL achieves superior top-1 localization and robust performance across crash types, with practical insights into confidence calibration and tool usage patterns. The work demonstrates the viability of LLM-based debugging agents in production environments while outlining key limitations and avenues for improvement in explanation reliability and deep-code navigation.

Abstract

Fault Localization (FL) aims to identify root causes of program failures. FL typically targets failures observed from test executions, and as such, often involves dynamic analyses to improve accuracy, such as coverage profiling or mutation testing. However, for large industrial software, measuring coverage for every execution is prohibitively expensive, making the use of such techniques difficult. To address these issues and apply FL in an industrial setting, this paper proposes AutoCrashFL, an LLM agent for the localization of crashes that only requires the crashdump from the Program Under Test (PUT) and access to the repository of the corresponding source code. We evaluate AutoCrashFL against real-world crashes of SAP HANA, an industrial software project consisting of more than 35 million lines of code. Experiments reveal that AutoCrashFL is more effective in localization, as it identified 30% crashes at the top, compared to 17% achieved by the baseline. Through thorough analysis, we find that AutoCrashFL has attractive practical properties: it is relatively more effective for complex bugs, and it can indicate confidence in its results. Overall, these results show the practicality of LLM agent deployment on an industrial scale.
Paper Structure (24 sections, 8 figures, 6 tables)

This paper contains 24 sections, 8 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Fault Localization
  4. LLM Agents for Automated Debugging
  5. AutoFL
  6. Crash Analysis
  7. Methodology
  8. Crashdump Parser
  9. Repository Interface
  10. AutoCrashFL
  11. Ranking Aggregation
  12. Experimental Setting
  13. Research Questions
  14. Dataset Collection
  15. AutoCrashFL Parameters
  16. ...and 9 more sections

Figures (8)

  • Figure 1: Diagram of AutoCrashFL.
  • Figure 2: Prompts used by AutoCrashFL, lightly truncated for presentation.
  • Figure 3: Rank Aggregation based on Voting
  • Figure 4: Ratio of buggy files whose base filenames explicitly appear in the CRASH_STACK. The green triangle denotes the mean value, while the orange line indicates the median.
  • Figure 5: Venn diagram of crash instances localized at the top by AutoCrashFL and Baseline 2
  • ...and 3 more figures