Mut4All: Fuzzing Compilers via LLM-Synthesized Mutators Learned from Bug Reports
Bo Wang, Pengyang Wang, Chong Chen, Ming Deng, Jieke Shi, Qi Sun, Chengran Yang, Youfang Lin, Zhou Yang, Junjie Chen, Jun Sun, David Lo
TL;DR
Mutation-based fuzzing benefits from realistic seed programs but historically relies on hand-designed mutators, limiting expressiveness and cross-language reuse. Mut4All introduces a fully automated, three-agent pipeline that learns mutator designs from historical bug reports, synthesizes implementations with language-specific templates, and refines them via compiler feedback, extended by adaptive seed enhancement. On Rust and C++, Mut4All synthesized 722 mutators (319 Rust, 403 C++) at roughly $0.08 per mutator, uncovering 96 bugs (58 new, 22 fixed) across rustc/gccrs and GCC/Clang, and achieving top coverage among baselines. The approach demonstrates strong cross-language applicability and cost-effectiveness for practical bug discovery, while highlighting areas for fidelity improvements and deprecation-aware generation to further enhance mutator quality and effectiveness.
Abstract
Mutation-based fuzzing is effective for uncovering compiler bugs, but designing high-quality mutators for modern languages with complex constructs (e.g., templates, macros) remains challenging. Existing methods rely heavily on manual design or human-in-the-loop correction, limiting scalability and cross-language generalizability. We present Mut4All, a fully automated, language-agnostic framework that synthesizes mutators using Large Language Models (LLMs) and compiler-specific knowledge from bug reports. It consists of three agents: (1) a mutator invention agent that identifies mutation targets and generates mutator metadata using compiler-related insights; (2) a mutator implementation synthesis agent, fine-tuned to produce initial implementations; and (3) a mutator refinement agent that verifies and corrects the mutators via unit-test feedback. Mut4All processes 1000 bug reports (500 Rust, 500 C++), yielding 319 Rust and 403 C++ mutators at ~$0.08 each via GPT-4o. Our customized fuzzer, using these mutators, finds 62 bugs in Rust compilers (38 new, 7 fixed) and 34 bugs in C++ compilers (16 new, 1 fixed). Mut4All outperforms existing methods in both unique crash detection and coverage, ranking first on Rust and second on C++.