Psyzkaller: Learning from Historical and On-the-Fly Execution Data for Smarter Seed Generation in OS kernel Fuzzing

TL;DR

Psyzkaller tackles the inefficiency of kernel fuzzing caused by invalid SDR-violating seed generation. It learns syscall dependency relations from both historic traces (via the DongTing dataset) and online fuzzing data using a lightweight Bigram model, then augments Syzkaller's seed-generation with an augmented Choice Table and a bidirectional RandomWalk strategy. The approach yields higher branch coverage and substantially more crashes, including eight previously unknown vulnerabilities, demonstrating the practical value of SDR-informed seed generation. This work shows how combining historical knowledge with online adaptation can improve kernel fuzzing efficiency and vulnerability discovery at scale.

Abstract

Fuzzing has become a cornerstone technique for uncovering vulnerabilities and enhancing the security of OS kernels. However, state-of-the-art kernel fuzzers, including the de facto standard Syzkaller, struggle to generate valid syscall sequences that respect implicit Syscall Dependency Relations (SDRs). Consequently, many generated seeds either fail kernel validation or cannot penetrate deep execution paths, resulting in significant inefficiency. We hypothesize that SDRs can be effectively learned from both historic and present kernel execution data, and that incorporating these learned relations into fuzzing can substantially improve seed validity and diversity. To validate this, we propose an approach that utilizes the N-gram model to mine SDRs from the Dongting dataset-one of the largest Linux kernel execution datasets available-as well as from execution traces collected on the fly during fuzzing. The resulting model is used to continuously augment the Choice Table of Syzkaller to improve its seed generation and demonstrably increases the Shannon Entropy of the Choice Table throughout fuzzing, reflecting more empirically-grounded choices in expanding syscall sequences into valid and diverse seeds. In addition, we introduce a Random Walk strategy that instructs Syzkaller to construct seeds in a bidirectional manner to further diversify the generated seeds. We implement our approach in a prototype, Psyzkaller, built on top of Syzkaller. Experiments on three representative Linux kernel versions show that Psyzkaller improves Syzkaller's code coverage by 4.6%-7.0% in 48-hour fuzzing, while triggering 110.4%-187.2% more crashes. Moreover, our investigation shows that Psyzkaller discovered eight previously unknown kernel vulnerabilities, compared to only one found by Syzkaller.

Figures (9)

• Figure 1: The impact of Bigram on Syskaller's Choice Table using the Shannon Index as a metrics
• Figure 2: System architecture of Psyzkaller.
• Figure 3: Branch Coverages Achieved by Psyzkaller under Different Weight Ratio Combinations in Training
• Figure 4: Crashes Triggered by Psyzkaller under Different Weight Ratio Combinations in Training
• Figure 5: Branch Coverage Achieved by Psyzkaller under Different Strategy Settings