Analytical Performance Estimation during Code Generation on Modern GPUs

TL;DR

The paper tackles the challenge of selecting effective code-generation configurations for GPUs by introducing an analytic performance estimator (Warpspeed) that builds a memory-hierarchy-aware extension of the roofline model. It derives cache- and memory-traffic metrics from high-level address expressions produced by code generators (e.g., pystencils) and couples these metrics to a ranking workflow that avoids kernel execution for hardware exploration. The approach is demonstrated on two challenging GPU workloads—a range-four 3D stencil and a complex LBM solver—revealing meaningful configuration rankings and insights into cache reuse through Layer Conditions, while highlighting limitations in capturing latency effects that limit top-rank accuracy. Overall, the method enables fast, hardware-aware exploration of large configuration spaces, supports architectural studies (e.g., V100 to A100 transitions), and can guide improvements in code-generation strategies.

Abstract

Automatic code generation is frequently used to create implementations of algorithms specifically tuned to particular hardware and application parameters. The code generation process involves the selection of adequate code transformations, tuning parameters, and parallelization strategies. We propose an alternative to time-intensive autotuning, scenario-specific performance models, or black-box machine learning to select the best-performing configuration. This paper identifies the relevant performance-defining mechanisms for memory-intensive GPU applications through a performance model coupled with an analytic hardware metric estimator. This enables a quick exploration of large configuration spaces to identify highly efficient code candidates with high accuracy. We examine the changes of the A100 GPU architecture compared to the predecessor V100 and address the challenges of how to model the data transfer volumes through the new memory hierarchy. We show how our method can be coupled to the pystencils stencil code generator, which is used to generate kernels for a range-four 3D-25pt stencil and a complex two-phase fluid solver based on the Lattice Boltzmann Method. For both, it delivers a ranking that can be used to select the best-performing candidate. The method is not limited to stencil kernels but can be integrated into any code generator that can generate the required address expressions.

Figures (25)

• Figure 1: Illustration of the performance prediction workflow
• Figure 2: Data transfer rate vs data set size for two streaming kernels on a repeating data set. LOAD: , SCALE: . Each thread's index is computed from the threads global thread index modulo the number of elements in the data set, resulting in repeated access to the same data by different threads.
• Figure 3: Data transfer rate vs thread count/occupancy for the SCALE kernel . The occupancy is artificially limited to two thread blocks per SM using a shared memory allocation as a spoiler. By adjusting the thread block size the occupancy is increased from $2\times32$ threads to full occupancy at $2\times1024$ threads per SM.
• Figure 4: Illustration of cache bank conflicts. Each column is a cache bank, the numbers below are the number of addresses for that cache bank.
• Figure 5: Simplified representation of the grid iteration function. The two example visitors compute the number of unique accessed 32B cache lines and the number of bank conflicts in the set of computed addresses.