Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UBfuzz: Finding Bugs in Sanitizer Implementations

Shaohua Li, Zhendong Su

TL;DR

UBfuzz tackles sanitizer false negatives for undefined behavior by combining a targeted UB program generator with a crash-site mapping test oracle to separate sanitizer bugs from compiler optimizations. The framework automatically generates UB-rich programs via Shadow Statement Insertion, then performs differential testing across GCC and LLVM sanitizers (ASan, UBSan, MSan) to identify discrepancies, validated by crash-site mapping. In five months, UBfuzz found 31 sanitizer FN bugs, with 20 confirmed and 6 fixed (all in GCC), highlighting substantial latent issues across optimization levels. The work demonstrates a practical, automated approach for sanitizer validation that can improve the reliability of UB detection in production compilers and informs ongoing sanitizer engineering.

Abstract

In this paper, we propose a testing framework for validating sanitizer implementations in compilers. Our core components are (1) a program generator specifically designed for producing programs containing undefined behavior (UB), and (2) a novel test oracle for sanitizer testing. The program generator employs Shadow Statement Insertion, a general and effective approach for introducing UB into a valid seed program. The generated UB programs are subsequently utilized for differential testing of multiple sanitizer implementations. Nevertheless, discrepant sanitizer reports may stem from either compiler optimization or sanitizer bugs. To accurately determine if a discrepancy is caused by sanitizer bugs, we introduce a new test oracle called crash-site mapping. We have incorporated our techniques into UBfuzz, a practical tool for testing sanitizers. Over a five-month testing period, UBfuzz successfully found 31 bugs in both GCC and LLVM sanitizers. These bugs reveal the serious false negative problems in sanitizers, where certain UBs in programs went unreported. This research paves the way for further investigation in this crucial area of study.

UBfuzz: Finding Bugs in Sanitizer Implementations

TL;DR

UBfuzz tackles sanitizer false negatives for undefined behavior by combining a targeted UB program generator with a crash-site mapping test oracle to separate sanitizer bugs from compiler optimizations. The framework automatically generates UB-rich programs via Shadow Statement Insertion, then performs differential testing across GCC and LLVM sanitizers (ASan, UBSan, MSan) to identify discrepancies, validated by crash-site mapping. In five months, UBfuzz found 31 sanitizer FN bugs, with 20 confirmed and 6 fixed (all in GCC), highlighting substantial latent issues across optimization levels. The work demonstrates a practical, automated approach for sanitizer validation that can improve the reliability of UB detection in production compilers and informs ongoing sanitizer engineering.

Abstract

In this paper, we propose a testing framework for validating sanitizer implementations in compilers. Our core components are (1) a program generator specifically designed for producing programs containing undefined behavior (UB), and (2) a novel test oracle for sanitizer testing. The program generator employs Shadow Statement Insertion, a general and effective approach for introducing UB into a valid seed program. The generated UB programs are subsequently utilized for differential testing of multiple sanitizer implementations. Nevertheless, discrepant sanitizer reports may stem from either compiler optimization or sanitizer bugs. To accurately determine if a discrepancy is caused by sanitizer bugs, we introduce a new test oracle called crash-site mapping. We have incorporated our techniques into UBfuzz, a practical tool for testing sanitizers. Over a five-month testing period, UBfuzz successfully found 31 bugs in both GCC and LLVM sanitizers. These bugs reveal the serious false negative problems in sanitizers, where certain UBs in programs went unreported. This research paves the way for further investigation in this crucial area of study.
Paper Structure (19 sections, 12 figures, 6 tables, 2 algorithms)

This paper contains 19 sections, 12 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Illustrative Examples
  3. UB Program Generation
  4. Crash-Site Mapping as the Test Oracle
  5. Approach
  6. UB Conditions and Shadow Statement
  7. UB Program Generator
  8. Expression Matching --- GetMatchedExpr($\cdot$)
  9. Program Profiling --- Profile($\cdot$)
  10. Shadow Statement Synthesis and Insertion --- SynShadowStmt($\cdot$) & Insert($\cdot$)
  11. Discussions
  12. Crash-site Mapping as the Test Oracle
  13. Empirical Evaluation
  14. Implementation and Evaluation Setup
  15. RQ1: Bug Finding
  16. ...and 4 more sections

Figures (12)

  • Figure 1: Line 8 in a.c contains a stack-buffer-overflow (left). GCC ASan at -O0 successfully detects it (top right). GCC's Asan at -O2, however, overlooks it (bottom right).
  • Figure 2: The high level compilation pipeline of ASan in GCC/gcc.
  • Figure 3: GCC -O2 optimizes away the UB code, thus ASan cannot discover the UB.
  • Figure 4: Code instrumentation for UB insertion.
  • Figure 5: Partial executed instructions in $b_c$ and $b_n$. The comment shows the corresponding line and offset in the source code.
  • ...and 7 more figures

Theorems & Definitions (2)

  • Definition 1: Execution Profile
  • Definition 2: Crash Site