Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bug Histories as Sources of Compiler Fuzzing Mutators

Lingjun Liu, Feiran Qin, Owolabi Legunsen, Marcelo d'Amorim

TL;DR

This work investigates whether compiler bug histories can be mined to generate effective fuzzing mutators. It introduces IssueMut, a two-part system that (i) mines mutators from GCC/LLVM bug reports by deriving positive and negative inputs and (ii) retrofits these mutators into a state-of-the-art mutational fuzzer (MetaMutou2024mutators). Across 24-hour campaigns, IssueMut discovers more unique crashes and reveals crashes missed by baselines, with 60 confirmed or fixed bugs reported to developers. The results demonstrate that bug-history mutators complement existing mutational strategies and can significantly boost coverage and bug detection in real-world compilers. The work offers practical lessons on leveraging bug histories, including focusing on recent language features (e.g., C23) and tree-level mutations, to accelerate bug discovery in compiler infrastructures.

Abstract

Bugs in compilers, which are critical infrastructure today, can have outsized negative impacts. Mutational fuzzers aid compiler bug detection by systematically mutating compiler inputs, i.e., programs. Their effectiveness depends on the quality of the mutators used. Yet, no prior work used compiler bug histories as a source of mutators. We propose IssueMut, the first approach for extracting compiler fuzzing mutators from bug histories. Our insight is that bug reports contain hints about program elements that induced compiler bugs; they can guide fuzzers towards similar bugs. IssueMut uses an automated method to mine mutators from bug reports and retrofit such mutators into existing mutational compiler fuzzers. Using IssueMut, we mine 587 mutators from 1760 GCC and LLVM bug reports. Then, we run IssueMut on these compilers, with all their test inputs as seed corpora. We find that "bug history" mutators are effective: they find new bugs that a state-of-the-art mutational compiler fuzzer misses-28 in GCC and 37 in LLVM. Of these, 60 were confirmed or fixed, validating our idea that bug histories have rich information that compiler fuzzers should leverage.

Bug Histories as Sources of Compiler Fuzzing Mutators

TL;DR

This work investigates whether compiler bug histories can be mined to generate effective fuzzing mutators. It introduces IssueMut, a two-part system that (i) mines mutators from GCC/LLVM bug reports by deriving positive and negative inputs and (ii) retrofits these mutators into a state-of-the-art mutational fuzzer (MetaMutou2024mutators). Across 24-hour campaigns, IssueMut discovers more unique crashes and reveals crashes missed by baselines, with 60 confirmed or fixed bugs reported to developers. The results demonstrate that bug-history mutators complement existing mutational strategies and can significantly boost coverage and bug detection in real-world compilers. The work offers practical lessons on leveraging bug histories, including focusing on recent language features (e.g., C23) and tree-level mutations, to accelerate bug discovery in compiler infrastructures.

Abstract

Bugs in compilers, which are critical infrastructure today, can have outsized negative impacts. Mutational fuzzers aid compiler bug detection by systematically mutating compiler inputs, i.e., programs. Their effectiveness depends on the quality of the mutators used. Yet, no prior work used compiler bug histories as a source of mutators. We propose IssueMut, the first approach for extracting compiler fuzzing mutators from bug histories. Our insight is that bug reports contain hints about program elements that induced compiler bugs; they can guide fuzzers towards similar bugs. IssueMut uses an automated method to mine mutators from bug reports and retrofit such mutators into existing mutational compiler fuzzers. Using IssueMut, we mine 587 mutators from 1760 GCC and LLVM bug reports. Then, we run IssueMut on these compilers, with all their test inputs as seed corpora. We find that "bug history" mutators are effective: they find new bugs that a state-of-the-art mutational compiler fuzzer misses-28 in GCC and 37 in LLVM. Of these, 60 were confirmed or fixed, validating our idea that bug histories have rich information that compiler fuzzers should leverage.

Paper Structure

This paper contains 38 sections, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Example
  3. Clang bug https://github.com/llvm/llvm-project/issues/120083
  4. Overview
  5. Positive and Negative Examples
  6. Mutator Creation
  7. Summary
  8. Approach
  9. Mutator Mining
  10. Scrape positive test case and issue data (❶)
  11. Extract negative test case (❷)
  12. Validate negative test case (❸)
  13. Generate mutators (❹)
  14. Enhanced Mutational Compiler Fuzzer
  15. Evaluation
  16. ...and 23 more sections

Figures (8)

  • Figure 1: The workflow of IssueMut's mutator miner.
  • Figure 2: Fragment of bug report issue108449 and corresponding positive and negative test cases.
  • Figure 3: Comparison of IssueMut against baselines on coverage (\ref{['fig:gcc-coverage']}), number of files (\ref{['tab:number-of-files']}), and crashes (\ref{['fig:venn-rq1']}).
  • Figure 4: Static characterization.
  • Figure 5: Dynamic characterization.
  • ...and 3 more figures