Evaluating Application Characteristics for GPU Portability Layer Selection

TL;DR

The paper investigates how different GPU portability layers (Kokkos, Alpaka, SYCL, OpenMP, std::par) perform across representative High Energy Physics workloads ported by the HEp-CCE. By analyzing kernel runtime and launch latency, concurrency, external library compatibility, memory layout, RNG/FFT/atomic support, compilation time, and runtime provisioning using testbeds from CMS, ATLAS, and DUNE, the study identifies task- and pattern-specific strengths and limitations of each layer. Key findings include substantial variability in launch latency, concurrency handling, and memory-transfer overheads, as well as notable challenges with external libraries and multi-architecture provisioning. The work provides practical guidelines to help developers select a GPU portability technology aligned with their code characteristics, while highlighting the ongoing rapid evolution of both portability frameworks and native compilers toward better cross-vendor performance and portability.

Abstract

GPUs have become the dominant source of computing power for high performance computing and are increasingly being used across the High Energy Physics computing landscape for a wide variety of tasks. Though NVIDIA is currently the main provider of GPUs, AMD and Intel are rapidly increasing their market share. As a result, programming using a vendor-specific language such as CUDA can significantly reduce deployment choices. There are a number of portability layers such as Kokkos, Alpaka, SYCL, OpenMP and std::par that permit execution on a broad range of GPU and CPU architectures, significantly increasing the flexibility of application programmers. However, each of these portability layers has its own characteristics, performing better at some tasks and worse at others, or placing limitations on aspects of the application. In this presentation, we report on a study of application and kernel characteristics that can influence the choice of a portability layer and show how each layer handles these characteristics. We have analyzed representative heterogeneous applications from CMS (patatrack and p2r), DUNE (Wire-Cell Toolkit), and ATLAS (FastCaloSim) to identify key application characteristics that have different behaviors for the various portability technologies. Using these results, developers can make more informed decisions on which GPU portability technology is best suited to their application.

Figures (1)

• Figure 1: Hardware support of portability layers. Dark green indicates full support, light green indicates partial support or that the project is still under development, orange is a proof of concept with an uncertain future, and red indicates no support (though there may have been support in the past).