Data-driven Test Generation for Fuzzing AI Compiler

TL;DR

AI compilers suffer bugs across model loading, high-level optimizations, and low-level hardware-specific optimizations, risking both reliability and model accuracy. The paper presents a unified data-driven testing framework with three components: OPERA for migrating model-loading tests, OATest for optimization-aware graph synthesis, and HARMONY for mutation-based low-level IR testing, aimed at stage-specific coverage. Empirical results show the framework uncovers 266 previously unknown bugs across multiple compilers and frontends (170, 56, and 40 for OPERA, OATest, and HARMONY respectively), demonstrating substantially improved coverage and bug discovery over prior fuzzers. This work provides a practical path toward more reliable AI compiler stacks and sets the stage for cross-compiler adaptation and automated debugging aids.

Abstract

Artificial Intelligence (AI) compilers are critical for efficiently deploying AI models across diverse hardware platforms. However, they remain prone to bugs that can compromise both compiler reliability and model correctness. Thus, ensuring the quality of AI compilers is crucial. In this work, we present a unified data-driven testing framework that systematically addresses stage-specific challenges in AI compilers. Specifically, OPERA migrates tests for AI libraries to test various operator conversion logic in the model loading stage. OATest synthesizes diverse optimization-aware computational graphs for testing high-level optimizations. HARMONY generates and mutates diverse low-level IR seeds to generate hardware-optimization-aware tests for testing low-level optimizations. Together, these techniques provide a comprehensive, stage-aware framework that enhances testing coverage and effectiveness, detecting 266 previously unknown bugs in four widely used AI compilers.