KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI Workloads

TL;DR

KPerfIR introduces a compiler-centric framework for GPU performance tooling by embedding profiling capabilities directly into the compiler IR, enabling region-based, fine-grained intra-kernel analysis. Implemented on the Triton MLIR-based stack, it provides a programmable, reusable instrumentation layer that lowers to vendor-specific backends while preserving high-level program semantics. The approach yields a region-based timing tool with low overhead, accurate trace replay, and demonstrated benefits in FA3 overlap optimization and memory analytics, validated on state-of-the-art GPUs with realistic AI workloads. This work signals a path toward an open, compiler-driven ecosystem where performance tools evolve with AI compilers and hardware, delivering actionable insights to both compiler developers and kernel designers.

Abstract

In this work, we propose KPerfIR, a novel multilevel compiler-centric infrastructure to enable the development of customizable, extendable, and portable profiling tools tailored for modern artificial intelligence (AI) workloads on modern GPUs. Our approach integrates profiling capabilities directly into the compiler workflow, allowing profiling functionalities to be implemented as compiler passes, offering a programmable and reusable framework for performance analysis. This design bridges the gap between compilers and profilers, enabling fine-grained insights into complex optimization challenges such as overlapping the execution of fine-grained function units on GPUs. KPerfIR is integrated into the Triton infrastructure to highlight the power of a compiler-centric approach to advance performance analysis and optimization in the ever-evolving landscape of AI compilers. Our evaluation shows that our tool incurs low overhead (8.2%), provides accurate measurements (2% relative error), and delivers actionable insights into complicated GPU intra-kernel optimizations.

Figures (15)

• Figure 1: Concept of the KPerfIR infrastructure and ecosystem for compiler-centric performance tool. (Left) Overview and comparison of KPerfIR's compiler-centric design and prior profiler designs. (Right) Demonstrative examples of novel performance tools facilitated by the compiler-centric design of KPerfIR.
• Figure 2: GPU overlapping techniques
• Figure 3: Motivating examples
• Figure 4: IR design and conversion passes
• Figure 5: An example of high-level record operations