MobFuzz: Adaptive Multi-objective Optimization in Gray-box Fuzzing

TL;DR

MobFuzz tackles the challenge of simultaneously optimizing multiple objectives in coverage-guided gray-box fuzzing by reframing the problem as a multi-player multi-armed bandit (MPMAB) and introducing NIC, an evolutionary algorithm that drives Pareto-front optimization within the fuzzing loop. The system adaptively selects objective combinations and allocates energy to seeds, while NIC explores the objective space to reveal seeds with the best trade-offs without adding substantial overhead. Empirical evaluation on 12 real-world programs and the MAGMA dataset shows MobFuzz outperforms single-objective baselines, achieves up to 107% improvements in objective values, reduces energy usage by up to 55%, and yields higher branch coverage and more unique bugs. The approach demonstrates strong potential for practical fuzzing improvements and can extend to more objectives with minimal architectural changes, offering a scalable path to deeper software vulnerability discovery.

Abstract

Coverage-guided gray-box fuzzing (CGF) is an efficient software testing technique. There are usually multiple objectives to optimize in CGF. However, existing CGF methods cannot successfully find the optimal values for multiple objectives simultaneously. In this paper, we propose a gray-box fuzzer for multi-objective optimization (MOO) called MobFuzz. We model the multi-objective optimization process as a multi-player multi-armed bandit (MPMAB). First, it adaptively selects the objective combination that contains the most appropriate objectives for the current situation. Second, our model deals with the power schedule, which adaptively allocates energy to the seeds under the chosen objective combination. In MobFuzz, we propose an evolutionary algorithm called NIC to optimize our chosen objectives simultaneously without incurring additional performance overhead. To prove the effectiveness of MobFuzz, we conduct experiments on 12 real-world programs and the MAGMA data set. Experiment results show that multi-objective optimization in MobFuzz outperforms single-objective fuzzing in the baseline fuzzers. In contrast to them, MobFuzz can select the optimal objective combination and increase the values of multiple objectives up to 107%, with at most a 55% reduction in the energy consumption. Moreover, MobFuzz has up to 6% more program coverage and finds 3x more unique bugs than the baseline fuzzers. The NIC algorithm has at least a 2x improvement with a performance overhead of approximately 3%.

Figures (10)

• Figure 1: The main fuzzing loop of MobFuzz. The sub-processes in different colors are our key approaches in MobFuzz.
• Figure 2: Demonstration of the MPMAB model. The rectangles indicate objective combinations. The circles indicate seeds. The rewards of the colored shapes are known. The depth of color of the seeds indicates the amount of energy allocated.
• Figure 3: Values of the objectives and the selected objective combinations within 60 minutes. The 3 lines indicate the values of the objectives. The background colors represent the selected objective combinations within each time interval.
• Figure 4: The $\mathtt{X}$-axis denotes different selection strategies. The $\mathtt{Y}$-axis is the ratio of objective values ($\mathtt{\frac{v'}{v_{ M}}}$) in different selection strategies ($\mathtt{v'}$) in contrast to our MPMAB selection ($\mathtt{v_{ M}}$). $\mathtt{\frac{v'}{v_{ M}}} < \mathtt{1.0}$ means the objective value of this selection is less than the MPMAB selection and vice versa.
• Figure 5: Comparison of our adaptive power schedule with non-adaptive and MobFuzz$^{ \rm{-M}}$ (MobFuzz without MPMAB) schedules in 1 hour. The $\mathtt{Y}$-axis shows the amount of energy allocated by the schedules. The green background color denotes that in this time interval, our adaptive schedule adjusts the energy according to the chosen objective combination and the non-adaptive schedule fails to.