Better Debugging: Combining Static Analysis and LLMs for Explainable Crashing Fault Localization

TL;DR

This work tackles the challenge of post-release crashes in framework-based apps by proposing CrashTracker, which combines framework-aware static analysis with LLM-assisted explanations. The core ideas are the Exception-thrown Summary (ETS) to capture framework-specific exception semantics and the Candidate Information Summary (CIS) to provide LLM-ready context, enabling precise localization and explainable reasoning. Evaluation on Android framework crashes demonstrates strong localization (MRR ≈ $0.91$) with compact candidate sets (mean around $6.35$ candidates) and substantial improvements in explanation quality (≈ $67.04\%$ user-satisfaction gain) over static-only approaches. The approach offers practical impact by delivering explainable, efficient debugging support for post-release framework crashes, with CrashTracker and data publicly released.

Abstract

Nowadays, many applications do not exist independently but rely on various frameworks or libraries. The frequent evolution and the complex implementation of framework APIs induce many unexpected post-release crashes. Starting from the crash stack traces, existing approaches either perform direct call graph (CG) tracing or construct datasets with similar crash-fixing records to locate buggy methods. However, these approaches are limited by the completeness of CG or dependent on historical fixing records. Moreover, they fail to explain the buggy candidates by revealing their relationship with the crashing point. To fill the gap, we propose an explainable crashing fault localization approach by combining static analysis and LLM techniques. Our primary insight is that understanding the semantics of exception-throwing statements in the framework code can help find and apprehend the buggy methods in the app code. Based on this idea, first, we design the exception-thrown summary (ETS) that describes the key elements related to each framework-specific exception and extract ETSs by performing static analysis. Then we make data-tracking of its key elements to identify and sort buggy candidates for the given crash. After that, we introduce LLMs to improve the explainability of the localization results. To construct effective LLM prompts, we design the candidate information summary (CIS) that describes multiple types of explanation-related contexts and then extract CISs via static analysis. We apply our approach to one typical scenario, i.e., locating Android framework-specific crashing faults, and implement a tool CrashTracker. For fault localization, it exhibited an overall MRR value of 0.91 in precision. For fault explanation, compared to the naive one produced by static analysis only, the LLM-powered explanation achieved a 67.04% improvement in users' satisfaction score.

Figures (8)

• Figure 1: Workflow of Our Approach
• Figure 2: Execution Trace and Crash Trace
• Figure 3: Motivating Example of Framework-specific Exception
• Figure 4: Overview of CrashTracker's Fault Localization
• Figure 5: Workflow of CrashTracker's Fault Explanation