MLIR-Forge: A Modular Framework for Language Smiths

TL;DR

MLIR-Forge introduces a modular framework that splits random program generation into language-specific puzzle pieces (OpGens/TypeGens) and reusable generation logic, enabling rapid creation of IR-specific program generators within MLIR. By embedding generation directly in MLIR via a constrained IR builder and standardized interfaces, it avoids translator-related brittleness and supports cross-dialect reuse. The authors demonstrate three Smiths (MLIR-Smith, SDFG-Smith, WASM-Smith) that uncover hundreds of bugs across MLIR, DaCe, and WebAssembly with modest development effort and modest hardware resources, underscoring the framework’s practicality. The work contributes a concrete pathway to scalable IR testing, enabling researchers to focus on IR design and optimization while maintaining high confidence in experimental results.

Abstract

Optimizing compilers are essential for the efficient and correct execution of software across various scientific fields. Domain-specific languages (DSL) typically use higher level intermediate representations (IR) in their compiler pipelines for domain-specific optimizations. As these IRs add to complexity, it is crucial to test them thoroughly. Random program generators have proven to be an effective tool to test compilers through differential and fuzz testing. However, developing specialized program generators for compiler IRs is not straightforward and demands considerable resources. We introduce MLIR-Forge, a novel random program generator framework that leverages the flexibility of MLIR, aiming to simplify the creation of specialized program generators. MLIR-Forge achieves this by splitting the generation process into fundamental building blocks that are language specific, and reusable program creation logic that constructs random programs from these building blocks. This hides complexity and furthermore, even the language specific components can be defined using a set of common tools. We demonstrate MLIR-Forge's capabilities by generating MLIR with built-in dialects, WebAssembly, and a data-centric program representation, DaCe -- requiring less than a week of development time in total for each of them. Using the generated programs we conduct differential testing and find 9 MLIR, 15 WebAssembly, and 774 DaCe groups of bugs with the corresponding program generators, after running them until the rate of new bugs stagnates.

Figures (12)

• Figure 1: Overview of an operation.
• Figure 2: Illustrations of MLIR concepts.
• Figure 3: Illustration of the IR specialization trend and the unification through the MLIR framework.
• Figure 4: Illustration of an example Csmith and Polygeist pipeline with potential drawbacks.
• Figure 5: Illustration of the components of a random program generator and the abstraction through MLIR-Forge. We consider the example of a random program generator for the SDFG dialect, but the components would be the same regardless of the IR.