Machine Learning for Actionable Warning Identification: A Comprehensive Survey

TL;DR

This survey addresses the challenge of noisy and often unactionable static analysis warnings by systematically reviewing ML‑based AWI approaches from 2000 to 2023. It distills a four‑stage workflow—warning dataset preparation, preprocessing, model construction, and evaluation—and classifies methods by warning output format (classification, ranking, combination). The analysis covers data sources (SCA types, languages, real‑world vs synthetic), feature extraction/selection, dataset rebalancing, modeling choices (TML, DL, PTMs), and evaluation practices, highlighting strengths, weaknesses, and gaps. Practical directions emphasize diverse data collection, improved labeling strategies, fine‑grained warning features, semi/unsupervised learning, and the potential of large language models to enhance AWI. Overall, the paper provides a comprehensive foundation for advancing ML‑driven AWI with reproducible methods and actionable guidance for researchers and practitioners.

Abstract

Actionable Warning Identification (AWI) plays a crucial role in improving the usability of static code analyzers. With recent advances in Machine Learning (ML), various approaches have been proposed to incorporate ML techniques into AWI. These ML-based AWI approaches, benefiting from ML's strong ability to learn subtle and previously unseen patterns from historical data, have demonstrated superior performance. However, a comprehensive overview of these approaches is missing, which could hinder researchers/practitioners from understanding the current process and discovering potential for future improvement in the ML-based AWI community. In this paper, we systematically review the state-of-the-art ML-based AWI approaches. First, we employ a meticulous survey methodology and gather 51 primary studies from 2000/01/01 to 2023/09/01. Then, we outline the typical ML-based AWI workflow, including warning dataset preparation, preprocessing, AWI model construction, and evaluation stages. In such a workflow, we categorize ML-based AWI approaches based on the warning output format. Besides, we analyze the techniques used in each stage, along with their strengths, weaknesses, and distribution. Finally, we provide practical research directions for future ML-based AWI approaches, focusing on aspects like data improvement (e.g., enhancing the warning labeling strategy) and model exploration (e.g., exploring large language models for AWI).

Figures (7)

• Figure 1: Distribution of primary studies. The blue (yellow) dashed lines represent the trends in the number of journal (conference) studies over years.
• Figure 2: Typical ML-based AWI workflow and the proposed RQs based on such a workflow.
• Figure 3: Distribution of the warning dataset preparation under three categories of ML-based AWI approaches.
• Figure 4: Distribution of the warning feature extraction under three categories of ML-based AWI approaches.
• Figure 5: Distribution of the warning feature selection and warning dataset rebalancing techniques under three categories of ML-based AWI approaches. In Fig. \ref{['fig:rq3-wdrebalance']}, "UO" is to use undersampling and oversampling for AWI. "UOCE" is to use four sampling techniques for AWI.