Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding and Detecting Annotation-Induced Faults of Static Analyzers

Huaien Zhang, Yu Pei, Shuyun Liang, Shin Hwei Tan

TL;DR

This paper presents the first comprehensive study of annotation-induced faults (AIF) by analyzing 246 issues in six open-source and popular static analyzers and developed an automated testing framework called AnnaTester based on three metamorphic relations originating from the findings.

Abstract

Static analyzers can reason about the properties and behaviors of programs and detect various issues without executing them. Hence, they should extract the necessary information to understand the analyzed program well. Annotation has been a widely used feature for different purposes in Java since the introduction of Java 5. Annotations can change program structures and convey semantics information without awareness of static analyzers, consequently leading to imprecise analysis results. This paper presents the first comprehensive study of annotation-induced faults (AIF) by analyzing 246 issues in six open-source and popular static analyzers (i.e., PMD, SpotBugs, CheckStyle, Infer, SonarQube, and Soot). We analyzed the issues' root causes, symptoms, and fix strategies and derived ten findings and some practical guidelines for detecting and repairing annotation-induced faults. Moreover, we developed an automated testing framework called AnnaTester based on three metamorphic relations originating from the findings. AnnaTester generated new tests based on the official test suites of static analyzers and unveiled 43 new faults, 20 of which have been fixed. The results confirm the value of our study and its findings.

Understanding and Detecting Annotation-Induced Faults of Static Analyzers

TL;DR

This paper presents the first comprehensive study of annotation-induced faults (AIF) by analyzing 246 issues in six open-source and popular static analyzers and developed an automated testing framework called AnnaTester based on three metamorphic relations originating from the findings.

Abstract

Static analyzers can reason about the properties and behaviors of programs and detect various issues without executing them. Hence, they should extract the necessary information to understand the analyzed program well. Annotation has been a widely used feature for different purposes in Java since the introduction of Java 5. Annotations can change program structures and convey semantics information without awareness of static analyzers, consequently leading to imprecise analysis results. This paper presents the first comprehensive study of annotation-induced faults (AIF) by analyzing 246 issues in six open-source and popular static analyzers (i.e., PMD, SpotBugs, CheckStyle, Infer, SonarQube, and Soot). We analyzed the issues' root causes, symptoms, and fix strategies and derived ten findings and some practical guidelines for detecting and repairing annotation-induced faults. Moreover, we developed an automated testing framework called AnnaTester based on three metamorphic relations originating from the findings. AnnaTester generated new tests based on the official test suites of static analyzers and unveiled 43 new faults, 20 of which have been fixed. The results confirm the value of our study and its findings.
Paper Structure (45 sections, 16 figures, 6 tables)

This paper contains 45 sections, 16 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Static Analyzer
  4. Java Annotation
  5. Empirical Study on Annotation-induced Faults
  6. Target Static Analyzers
  7. Data Collection
  8. Data Labeling and Reliability Analysis
  9. RQ1: AIF Prone Annotations
  10. RQ2: Root Causes
  11. Incomplete Semantics (IS)
  12. Improper AST Traversal (IAT)
  13. Unrecognized Equivalent Annotations (UEA)
  14. Erroneous Type Operations (ETO)
  15. Incorrect AST Generation (IAG)
  16. ...and 30 more sections

Figures (16)

  • Figure 1: An annotated Java program that will cause PMD to crash.
  • Figure 2: The general workflow of a static analyzer
  • Figure 3: The number of issues induced by each annotation from the top 30 most AIF-prone annotations.
  • Figure 4: An incomplete semantics example in SONARQUBE-3804.
  • Figure 5: SONARQUBE-3536 sonar3536: A false negative caused by UEA
  • ...and 11 more figures

Theorems & Definitions (4)

  • Definition 1: Analysis Equivalence
  • Definition 2: MR1
  • Definition 3: MR2
  • Definition 4: MR3