Many Tools, Few Exploitable Vulnerabilities: A Survey of 246 Static Code Analyzers for Security

TL;DR

It is observed that most analyzers focus on a limited set of weaknesses, that the vulnerabilities they detect are rarely exploitable, and that evaluations use custom benchmarks that are too small to enable robust assessment.

Abstract

Static security analysis is a widely used technique for detecting software vulnerabilities across a wide range of weaknesses, application domains, and programming languages. While prior work surveyed static analyzes for specific weaknesses or application domains, no overview of the entire security landscape exists. We present a systematic literature review of 246 static security analyzers concerning their targeted vulnerabilities, application domains, analysis techniques, evaluation methods, and limitations. We observe that most analyzers focus on a limited set of weaknesses, that the vulnerabilities they detect are rarely exploitable, and that evaluations use custom benchmarks that are too small to enable robust assessment.

Figures (16)

• Figure 1: Programming languages and application domains targeted by static security analyzers
• Figure 2: Further objectives of static security analyzers
• Figure 3: Weaknesses scanned by static security analyzers organized by top-level CWE comprehensive categories
• Figure 4: Weaknesses scanned by static security analyzers
• Figure 5: Security features related to static security analyzers