An Empirical Study on the Distance Metric in Guiding Directed Grey-box Fuzzing

TL;DR

The first empirical study to explore how different distance metrics perform in guiding DGFs is conducted and highlights the inherent limitation of existing mutation strategies in generating high-quality testcases, calling for designing effective mutation strategies for directed fuzzing.

Abstract

Directed grey-box fuzzing (DGF) aims to discover vulnerabilities in specific code areas efficiently. Distance metric, which is used to measure the quality of seed in DGF, is a crucial factor in affecting the fuzzing performance. Despite distance metrics being widely applied in existing DGF frameworks, it remains opaque about how different distance metrics guide the fuzzing process and affect the fuzzing result in practice. In this paper, we conduct the first empirical study to explore how different distance metrics perform in guiding DGFs. Specifically, we systematically discuss different distance metrics in the aspect of calculation method and granularity. Then, we implement different distance metrics based on AFLGo. On this basis, we conduct comprehensive experiments to evaluate the performance of these distance metrics on the benchmarks widely used in existing DGF-related work. The experimental results demonstrate the following insights. First, the difference among different distance metrics with varying methods of calculation and granularities is not significant. Second, the distance metrics may not be effective in describing the difficulty of triggering the target vulnerability. In addition, by scrutinizing the quality of testcases, our research highlights the inherent limitation of existing mutation strategies in generating high-quality testcases, calling for designing effective mutation strategies for directed fuzzing. We open-source the implementation code and experiment dataset to facilitate future research in DGF.

Figures (5)

• Figure 1: Difference among arithmetic mean distance, harmonic mean distance, and closest target distance. Node distance is shown in the white circles. The target nodes are marked in gray.
• Figure 2: Lineage of PoC on different CVEs in the benchmark. Red 'x' nodes represent all the ancestor seeds of the PoC, with the rightmost red 'x' corresponding to the PoC itself. Grey nodes represent testcases derived from ancestor seeds of PoC. The x-axis represents the time (ms) when the seed or testcase is generated and the y-axis represents their distance.
• Figure 3: The Distribution of Lineage Length of PoC.
• Figure 4: Distance decrease distribution on different CVEs in the benchmark. The y-axis is the distance decrease while the x-axis is the time (ms) the seed mutates to the testcase.
• Figure 5: The cactus plot of distance decrease on different CVEs in the benchmark. The y-axis is the distance decrease while the x-axis is the number of mutation operations.