Tumbling Down the Rabbit Hole: How do Assisting Exploration Strategies Facilitate Grey-box Fuzzing?

TL;DR

The paper addresses the challenge of comprehensively evaluating assisting exploration strategies for grey-box fuzzing on a common benchmark. It demonstrates that dictionary-based strategies are highly effective and introduces CDFuzz, a lightweight, per-seed customized dictionary fuzzer that significantly improves edge coverage and bug discovery. The study reveals that many gains come from targeting equality constraints, while constraint-solvers face depth-related constraints and taint-tracking limitations; the proposed approach yields robust performance across diverse benchmarks. Practically, the work provides evidence and a concrete technique to enhance grey-box fuzzing, leading to more efficient vulnerability discovery in real-world software.

Abstract

Many assisting exploration strategies have been proposed to assist grey-box fuzzers in exploring program states guarded by tight and complex branch conditions such as equality constraints. Although they have shown promising results in their original papers, their evaluations seldom follow equivalent protocols, e.g., they are rarely evaluated on identical benchmarks. Moreover, there is a lack of sufficient investigations on the specifics of the program states explored by these strategies which can obfuscate the future application and development of such strategies. Consequently, there is a pressing need for a comprehensive study of assisting exploration strategies on their effectiveness, versatility, and limitations to enlighten their future development. To this end, we perform the first comprehensive study about the assisting exploration strategies for grey-box fuzzers. Specifically, we first collect nine recent fuzzers representing the mainstream assisting exploration strategies as our studied subjects and 21 real-world projects to form our benchmark suite. After evaluating the subjects on the benchmark suite, we then surprisingly find that the dictionary strategy is the most promising since it not only achieves similar or even slightly better performance over the other studied assisting exploration strategies in terms of exploring program states but also is more practical to be enhanced. Accordingly, we propose CDFUZZ, which generates a customized dictionary for each seed upon the baseline fuzzer AFL to improve over the original dictionary strategy. The evaluation results demonstrate that CDFUZZ increases the edge coverage by 16.1% on average for all benchmark projects over the best performer in our study (i.e., AFL++ with the dictionary strategy). CDFUZZ also successfully exposed 37 previously unknown bugs, with nine confirmed and seven fixed by the corresponding developers.

Figures (8)

• Figure 1: The workflow of assisting exploration strategies
• Figure 2: Ratios of the total number of equality constraints to the total number of unsatisfiable critical constraints
• Figure 3: Ratios of the execution time and explored edges of the input-to-state correspondence strategy
• Figure 4: An input-to-state fuzzing strategy failure case in xmllint parser.c
• Figure 5: A case in xmllint encoding.c