SmartFL: Semantics Based Probabilistic Fault Localization

TL;DR

SmartFL presents a semantics-based probabilistic fault localization framework that models only the correctness of program values rather than full semantics, enabling scalable inference. By constructing a dynamic dependency graph from test executions and converting it to a Bayesian network, it computes posterior probabilities for statements being faulty via loopy belief propagation, while applying techniques like adaptive folding, partial tracing, and loop compression to scale to large real-world programs. Evaluated on Defects4J 2.0, SmartFL achieves state-of-the-art statement-level accuracy (top-1 up to 14% and superior top-k performance) with significantly lower time costs (~205 seconds per fault) than MBFL and faster than SBFL, and it further boosts performance when combined with CombineFL. The work demonstrates that a principled, semantics-aware probabilistic model can deliver precise fault localization at scale and is complementary to existing techniques, offering practical impact for real-world debugging workflows.

Abstract

Testing-based fault localization has been a research focus in software engineering in the past decades. It localizes faulty program elements based on a set of passing and failing test executions. Since whether a fault could be triggered and detected by a test is related to program semantics, it is crucial to model program semantics in fault localization approaches. Existing approaches either consider the full semantics of the program (e.g., mutation-based fault localization and angelic debugging), leading to scalability issues, or ignore the semantics of the program (e.g., spectrum-based fault localization), leading to imprecise localization results. Our key idea is: by modeling only the correctness of program values but not their full semantics, a balance could be reached between effectiveness and scalability. To realize this idea, we introduce a probabilistic model by efficient approximation of program semantics and several techniques to address scalability challenges. Our approach, SmartFL(SeMantics bAsed pRobabilisTic Fault Localization), is evaluated on a real-world dataset, Defects4J 2.0. The top-1 statement-level accuracy of our approach is {14\%}, which improves 130\% over the best SBFL and MBFL methods. The average time cost is {205} seconds per fault, which is half of SBFL methods. After combining our approach with existing approaches using the CombineFL framework, the performance of the combined approach is significantly boosted by an average of 10\% on top-1, top-3, and top-5 accuracy compared to state-of-the-art combination methods.

Figures (4)

• Figure 1: A Motivating Example for Condition Modeling.
• Figure 2: Generated Bayesian Network for Figure \ref{['fig:example']}.
• Figure 3: Approach Workflow.
• Figure 4: Example Program Demonstrating Control Dependencies.

Theorems & Definitions (1)

• Definition 4.1: Dynamic Dependency Graph