Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tracking the Evolution of Static Code Warnings: the State-of-the-Art and a Better Approach

Junjie Li, Jinqiu Yang

TL;DR

This work tackles the problem of tracking how static code warnings evolve over software history, addressing limitations of the state-of-the-art tracking approach. It introduces StaticTracker, which adds refactoring-aware matching, Hungarian assignment for robust pair selection, and volatile-identifier handling to distinguish fixed vs non-fixed removals. The authors build a ground-truth dataset of 3,451 warnings across four Java projects, demonstrating that the SOTA approach yields only about 66% precision, while StaticTracker achieves about 90% precision on the same data and 90.2% in a broader generalization study. The results indicate that precise warning-tracking can significantly improve the effectiveness of static analysis in development workflows and downstream research such as automatic program repair and warning prioritization.

Abstract

Static bug detection tools help developers detect problems in the code, including bad programming practices and potential defects. Recent efforts to integrate static bug detectors in modern software development workflows, such as in code review and continuous integration, are shown to better motivate developers to fix the reported warnings on the fly. A proper mechanism to track the evolution of the reported warnings can better support such integration. Moreover, tracking the static code warnings will benefit many downstream software engineering tasks, such as learning the fix patterns for automated program repair, and learning which warnings are of more interest, so they can be prioritized automatically. In addition, the utilization of tracking tools enables developers to concentrate on the most recent and actionable static warnings rather than being overwhelmed by the thousands of warnings from the entire project. This, in turn, enhances the utilization of static analysis tools. Hence, precisely tracking the warnings by static bug detectors is critical to improving the utilization of static bug detectors further.

Tracking the Evolution of Static Code Warnings: the State-of-the-Art and a Better Approach

TL;DR

This work tackles the problem of tracking how static code warnings evolve over software history, addressing limitations of the state-of-the-art tracking approach. It introduces StaticTracker, which adds refactoring-aware matching, Hungarian assignment for robust pair selection, and volatile-identifier handling to distinguish fixed vs non-fixed removals. The authors build a ground-truth dataset of 3,451 warnings across four Java projects, demonstrating that the SOTA approach yields only about 66% precision, while StaticTracker achieves about 90% precision on the same data and 90.2% in a broader generalization study. The results indicate that precise warning-tracking can significantly improve the effectiveness of static analysis in development workflows and downstream research such as automatic program repair and warning prioritization.

Abstract

Static bug detection tools help developers detect problems in the code, including bad programming practices and potential defects. Recent efforts to integrate static bug detectors in modern software development workflows, such as in code review and continuous integration, are shown to better motivate developers to fix the reported warnings on the fly. A proper mechanism to track the evolution of the reported warnings can better support such integration. Moreover, tracking the static code warnings will benefit many downstream software engineering tasks, such as learning the fix patterns for automated program repair, and learning which warnings are of more interest, so they can be prioritized automatically. In addition, the utilization of tracking tools enables developers to concentrate on the most recent and actionable static warnings rather than being overwhelmed by the thousands of warnings from the entire project. This, in turn, enhances the utilization of static analysis tools. Hence, precisely tracking the warnings by static bug detectors is critical to improving the utilization of static bug detectors further.
Paper Structure (17 sections, 12 figures, 10 tables, 3 algorithms)

This paper contains 17 sections, 12 figures, 10 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Motivating Examples and Background
  3. Challenges of tracking static code warnings in development history
  4. The Metadata of Static Code Warnings
  5. The state-of-the-art (SOTA) solution
  6. Examining the performance of the SOTA Approach
  7. Studied Subjects
  8. Crafting the Dataset with Manual Labeling
  9. Evaluating the Performance of the SOTA Approach
  10. Investigating the Inaccuracies of the SOTA Approach
  11. StaticTracker: A Better Approach to Track Static Warnings
  12. Evaluation of StaticTracker
  13. Threats to validity
  14. External Validity
  15. Internal Validity
  16. ...and 2 more sections

Figures (12)

  • Figure 1: An overview of our study.
  • Figure 2: An example to show how the SOTA approach may produce false positives and false negatives due to incorrect mappings. Note that the SOTA approach only reports the combined status removed rather than removedfix and removednon-fix separately.
  • Figure 3: An example of the representation of one static code warning from Spotbugs. Note that the representation has been simplified to only show the information used by the SOTA matching approach.
  • Figure 4: An example to show how the location-based matching works to match the two "NullAssignment" despite the different line numbers.
  • Figure 5: An example of false positives due to method renaming.
  • ...and 7 more figures