GPU Kernel Optimization Beyond Full Builds: An LLM Framework with Minimal Executable Programs

TL;DR

High-performance computing often stalls on hotspot GPU kernels, and full-application builds for performance feedback are prohibitively expensive. The authors present an end-to-end LLM-based framework that constructs Minimal Executable Programs (MEPs), uses a performance-feedback iterative loop with a trimmed-mean of $R$ runs, and enforces $T_{\text{ker}} \ge T_{\text{min}}$ and $T_{\text{overall}} \le T_{\text{max}}$, augmented by Automatic Error Repair and Functional Equivalence with Performance Pattern Inheritance. The approach demonstrates cross-platform portability (CUDA/NVIDIA and HIP/DCU) and delivers integrated speedups over direct LLM optimization across PolyBench, AMD APP SDK, and large-scale HPC kernels, while requiring no full-source dependencies. Overall, the framework reduces validation costs for GPU kernel optimization and offers practical, low-cost improvements, with future work aimed at improving environment fidelity and adaptive integration.

Abstract

In high-performance computing, hotspot GPU kernels are primary bottlenecks, and expert manual tuning is costly and hard to port. Large language model methods often assume kernels can be compiled and executed cheaply, which fails in large applications where full builds and runs are expensive. We present an end-to-end LLM framework with performance feedback that optimizes kernels without building the full application. From independently extracted hotspot kernels, it automatically completes code into a Minimal Executable Program (MEP), then performs multi-round iterative optimization and evaluation outside the full application. The framework integrates Automatic Error Repair and Performance Pattern Inheritance to fix faults, preserve correctness, reuse effective tiling/memory/synchronization strategies, and reduce search cost. Optimized variants are reintegrated into the original application for validation. We evaluate on NVIDIA GPUs and the Haiguang Deep Computing Unit (DCU) platform (AMD-licensed architecture) using PolyBench, the AMD APP SDK, and hotspot kernels from large-scale supercomputing applications. The method achieves average speedups of 5.05x (PolyBench on NVIDIA), 7.77x (PolyBench on DCU), 1.77x (AMD APP SDK), and 1.25x on three hotspot kernels, surpassing direct LLM optimization. The approach requires no full-source dependencies, offers cross-platform portability, and enables practical, low-cost GPU kernel optimization.

Figures (3)

• Figure 1: The MEP-based LLM framework for GPU kernel optimization.
• Figure 2: The framework of MEP construction and evaluation.
• Figure 3: The framework of performance-feedback iterative optimization.