Evaluation of Programming Models and Performance for Stencil Computation on Current GPU Architectures

TL;DR

The paper evaluates high-order stencil kernels on NVIDIA A100 and GH200 GPUs using CUDA, OpenACC, and OpenMP target offloading. It analyzes a $25$-point, $8^{th}$-order stencil for wave-equation simulations, investigates Hopper-specific features like thread block clusters, and provides architecture-aware optimizations and recommendations. It compares performance, portability, and power across programming models and GPU generations, reporting up to 58% performance gains on GH200 and up to 30% improvements from asynchronous strategies for OpenACC/OpenMP. The findings inform developers about when to prioritize raw performance versus portability in contemporary HPC workloads.

Abstract

Accelerated computing is widely used in high-performance computing. Therefore, it is crucial to experiment and discover how to better utilize GPUGPUs latest generations on relevant applications. In this paper, we present results and share insights about highly tuned stencil-based kernels for NVIDIA Ampere (A100) and Hopper (GH200) architectures. Performance results yield useful insights into the behavior of this type of algorithms for these new accelerators. This knowledge can be leveraged by many scientific applications which involve stencils computations. Further, evaluation of three different programming models: CUDA, OpenACC, and OpenMP target offloading is conducted on aforementioned accelerators. We extensively study the performance and portability of various kernels under each programming model and provide corresponding optimization recommendations. Furthermore, we compare the performance of different programming models on the mentioned architectures. Up to 58% performance improvement was achieved against the previous GPGPU's architecture generation for an highly optimized kernel of the same class, and up to 42% for all classes. In terms of programming models, and keeping portability in mind, optimized OpenACC implementation outperforms OpenMP implementation by 33%. If portability is not a factor, our best tuned CUDA implementation outperforms the optimized OpenACC one by 2.1x.

Figures (12)

• Figure 1: Thread-block-to-thread-block data exchange
• Figure 2: Data domain decomposition.
• Figure 3: Blocking strategies: (left) 3D blocking, (right) 2.5D blocking.
• Figure 4: Roofline plot of CUDA kernels on A100
• Figure 5: Roofline plot of CUDA kernels on GH200