Selecting Initial Seeds for Better JVM Fuzzing
Tianchang Gao, Junjie Chen, Dong Wang, Yile Guo, Yingquan Zhao, Zan Wang
TL;DR
This paper tackles seed selection in JVM fuzzing, where redundancy among initial seeds can hinder effectiveness within time budgets. It designs 10 seed selection methods spanning coverage-based, prefuzz-based, and program-feature-based approaches, and rigorously evaluates them on three JVM implementations using JavaTailor and VECT. The findings show that CFG-based program-feature methods (FISS_CFG) deliver the best performance and efficiency, while incorporating programs from open-source corpora further enhances fuzzing by revealing new behaviors and unknown bugs; 21 of 25 unknown bugs were confirmed or fixed by developers. The work demonstrates the practical impact of task-specific seed selection for JVM fuzzing and suggests avenues for adaptive budgets, fine-tuning code representations, and mixing diverse corpora to maximize defect discovery.
Abstract
Literature in traditional program fuzzing has confirmed that effectiveness is largely impacted by redundancy among initial seeds, thereby proposing a series of seed selection methods. JVM fuzzing, compared to traditional ones, presents unique characteristics, including large-scale and intricate code, and programs with both syntactic and semantic features. However, it remains unclear whether the existing seed selection methods are suitable for JVM fuzzing and whether utilizing program features can enhance effectiveness. To address this, we devise a total of 10 initial seed selection methods, comprising coverage-based, prefuzz-based, and program-feature-based methods. We then conduct an empirical study on three JVM implementations to extensively evaluate the performance of the seed selection methods within two SOTA fuzzing techniques (JavaTailor and VECT). Specifically, we examine performance from three aspects: (i) effectiveness and efficiency using widely studied initial seeds, (ii) effectiveness using the programs in the wild, and (iii) the ability to detect new bugs. Evaluation results first show that the program-feature-based method that utilizes the control flow graph not only has a significantly lower time overhead (i.e., 30s), but also outperforms other methods, achieving 142% to 269% improvement compared to the full set of initial seeds. Second, results reveal that the initial seed selection greatly improves the quality of wild programs and exhibits complementary effectiveness by detecting new behaviors. Third, results demonstrate that given the same testing period, initial seed selection improves the JVM fuzzing techniques by detecting more unknown bugs. Particularly, 21 out of the 25 detected bugs have been confirmed or fixed by developers. This work takes the first look at initial seed selection in JVM fuzzing, confirming its importance in fuzzing effectiveness and efficiency.