Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sidekick compilation with xDSL

Mathieu Fehr, Michel Weber, Christian Ulmann, Alexandre Lopoukhine, Martin Lücke, Théo Degioanni, Michel Steuwer, Tobias Grosser

TL;DR

The paper introduces sidekick compiler frameworks, with xDSL as a Python-based sidekick to MLIR, to bridge rapid prototyping and production-ready compilation. By sharing core concepts (e.g., SSA, nested regions, attributes) and exchanging IR definitions via the IRDL dialect, xDSL enables flexible workflows across teaching, DSL design, and MLIR feature prototyping while remaining interoperable with MLIR. Three use cases demonstrate gains in accessibility, design-space exploration, and performance potential, and quantitative comparisons show favorable startup, installation, and incremental-build profiles for xDSL relative to MLIR, at the expense of some runtime speed in large-scale parsing tasks. The IRDL-based sharing ecosystem and Python-centric approach offer a practical path toward a broader, more interconnected compiler ecosystem that supports rapid experimentation and smooth transitions to production.

Abstract

Traditionally, compiler researchers either conduct experiments within an existing production compiler or develop their own prototype compiler; both options come with trade-offs. On one hand, prototyping in a production compiler can be cumbersome, as they are often optimized for program compilation speed at the expense of software simplicity and development speed. On the other hand, the transition from a prototype compiler to production requires significant engineering work. To bridge this gap, we introduce the concept of sidekick compiler frameworks, an approach that uses multiple frameworks that interoperate with each other by leveraging textual interchange formats and declarative descriptions of abstractions. Each such compiler framework is specialized for specific use cases, such as performance or prototyping. Abstractions are by design shared across frameworks, simplifying the transition from prototyping to production. We demonstrate this idea with xDSL, a sidekick for MLIR focused on prototyping and teaching. xDSL interoperates with MLIR through a shared textual IR and the exchange of IRs through an IR Definition Language. The benefits of sidekick compiler frameworks are evaluated by showing on three use cases how xDSL impacts their development: teaching, DSL compilation, and rewrite system prototyping. We also investigate the trade-offs that xDSL offers, and demonstrate how we simplify the transition between frameworks using the IRDL dialect. With sidekick compilation, we envision a future in which engineers minimize the cost of development by choosing a framework built for their immediate needs, and later transitioning to production with minimal overhead.

Sidekick compilation with xDSL

TL;DR

The paper introduces sidekick compiler frameworks, with xDSL as a Python-based sidekick to MLIR, to bridge rapid prototyping and production-ready compilation. By sharing core concepts (e.g., SSA, nested regions, attributes) and exchanging IR definitions via the IRDL dialect, xDSL enables flexible workflows across teaching, DSL design, and MLIR feature prototyping while remaining interoperable with MLIR. Three use cases demonstrate gains in accessibility, design-space exploration, and performance potential, and quantitative comparisons show favorable startup, installation, and incremental-build profiles for xDSL relative to MLIR, at the expense of some runtime speed in large-scale parsing tasks. The IRDL-based sharing ecosystem and Python-centric approach offer a practical path toward a broader, more interconnected compiler ecosystem that supports rapid experimentation and smooth transitions to production.

Abstract

Traditionally, compiler researchers either conduct experiments within an existing production compiler or develop their own prototype compiler; both options come with trade-offs. On one hand, prototyping in a production compiler can be cumbersome, as they are often optimized for program compilation speed at the expense of software simplicity and development speed. On the other hand, the transition from a prototype compiler to production requires significant engineering work. To bridge this gap, we introduce the concept of sidekick compiler frameworks, an approach that uses multiple frameworks that interoperate with each other by leveraging textual interchange formats and declarative descriptions of abstractions. Each such compiler framework is specialized for specific use cases, such as performance or prototyping. Abstractions are by design shared across frameworks, simplifying the transition from prototyping to production. We demonstrate this idea with xDSL, a sidekick for MLIR focused on prototyping and teaching. xDSL interoperates with MLIR through a shared textual IR and the exchange of IRs through an IR Definition Language. The benefits of sidekick compiler frameworks are evaluated by showing on three use cases how xDSL impacts their development: teaching, DSL compilation, and rewrite system prototyping. We also investigate the trade-offs that xDSL offers, and demonstrate how we simplify the transition between frameworks using the IRDL dialect. With sidekick compilation, we envision a future in which engineers minimize the cost of development by choosing a framework built for their immediate needs, and later transitioning to production with minimal overhead.
Paper Structure (45 sections, 15 figures)

This paper contains 45 sections, 15 figures.

Table of Contents

  1. Introduction
  2. Sidekick Compilation
  3. Three Use Cases for SideKick Compilation
  4. Use case 1: Teaching compilation with ChocoPy
  5. User
  6. Needs
  7. Existing Workflows
  8. The xDSL Approach
  9. Use case 2: Designing a DSL-compiler
  10. User
  11. Existing Workflows
  12. The xDSL Approach
  13. Use case 3: Prototyping new MLIR features
  14. User
  15. Needs
  16. ...and 30 more sections

Figures (15)

  • Figure 1: Compiler frameworks that share the same core compilation concepts can offer tailored projections of these concepts for specific use cases while being still compatible through IRs and declarative IR definitions.
  • Figure 2: Dialect definitions (grey shapes in xDSL and MLIR) can be shared between xDSL and MLIR (dark-grey shapes exist in both). This enables the exploration of new workflows in the compiler space: python-native development of end-to-end compilers, pairing high-level DSLs with low-level compilers, and extending low-level compilers to explore new compiler design ideas.
  • Figure 3: A xDSL-only flow.
  • Figure 4: From xDSL to MLIR
  • Figure 5: Our Four-Level Stack DSL outperforms the state-of-the-art database DuckDB on the single-threaded Q6 TPC-H benchmark.
  • ...and 10 more figures