On Interaction Effects in Greybox Fuzzing

TL;DR

This work investigates whether the order of mutation operators in greybox fuzzing affects effectiveness. The authors provide empirical evidence of interaction effects between mutators and introduce MuoFuzz, a two-phase fuzzer that learns the conditional probability of the next mutator given the current one and uses a Markov-like random walk to generate sequences. Through extensive experiments on FuzzBench and MAGMA, MuoFuzz achieves higher code coverage than AFL++ and MOPT in most programs and finds several bugs that baselines miss, with ablations confirming the importance of modeling mutator interactions. The findings suggest that leveraging mutator interactions can meaningfully improve fuzzing performance and prompt future research into sequence-based mutator models and more nuanced training strategies. Overall, the paper demonstrates a practical approach to enhancing greybox fuzzing by conditioning mutator choice on prior context, yielding tangible gains in coverage and bug discovery.

Abstract

A greybox fuzzer is an automated software testing tool that generates new test inputs by applying randomly chosen mutators (e.g., flipping a bit or deleting a block of bytes) to a seed input in random order and adds all coverage-increasing inputs to the corpus of seeds. We hypothesize that the order in which mutators are applied to a seed input has an impact on the effectiveness of greybox fuzzers. In our experiments, we fit a linear model to a dataset that contains the effectiveness of all possible mutator pairs and indeed observe the conjectured interaction effect. This points us to more efficient fuzzing by choosing the most promising mutator sequence with a higher likelihood. We propose MuoFuzz, a greybox fuzzer that learns and chooses the most promising mutator sequences. MuoFuzz learns the conditional probability that the next mutator will yield an interesting input, given the previously selected mutator. Then, it samples from the learned probability using a random walk to generate mutator sequences. We compare the performance of MuoFuzz to AFL++, which uses a fixed selection probability, and MOPT, which optimizes the selection probability of each mutator in isolation. Experimental results on the FuzzBench and MAGMA benchmarks show that MuoFuzz achieves the highest code coverage and finds four bugs missed by AFL++ and one missed by both AFL++ and MOPT.

Figures (2)

• Figure 1: The left figure shows the number of interesting inputs generated by the mutator sequence $\langle i,j \rangle$, where $i$ is the mutator in the row and $j$ is the mutator in the column. The conditional probability on the right figure is derived by dividing each row of the left figure by its sum. As such, each row $i$ of the right matrix is a probability distribution, denoting how probable is that selecting $j$ as the second mutator will yield an interesting input, given that the first mutator is $i$. These concern the freetype target program.
• Figure 2: High-level overview of MuoFuzz.