Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs

TL;DR

This work presents a SYCL-based, fully-fused MLP implementation for the Intel Data Center GPU Max 1550 that maximizes data reuse in registers and shared memory by fusing layer computations. A roofline analysis demonstrates that the fusion increases arithmetic intensity and moves the computation toward being compute-bound, yielding substantial inference speedups and solid training performance when compared with CUDA and PyTorch baselines. The authors validate the approach across four domains—non-linear function approximation, image compression, Neural Radiance Fields, and Physics-Informed Neural Networks—showing up to 2.84x inference and 1.75x training improvements over a CUDA implementation, and up to 30x over PyTorch. The work provides an open-source path (tiny-dpcpp-nn) for accelerating MLP workloads on Intel GPUs, with clear directions for further optimization and broader data-type support.

Abstract

This paper presents a SYCL implementation of Multi-Layer Perceptrons (MLPs), which targets and is optimized for the Intel Data Center GPU Max 1550. To increase the performance, our implementation minimizes the slow global memory accesses by maximizing the data reuse within the general register file and the shared local memory by fusing the operations in each layer of the MLP. We show with a simple roofline model that this results in a significant increase in the arithmetic intensity, leading to improved performance, especially for inference. We compare our approach to a similar CUDA implementation for MLPs and show that our implementation on the Intel Data Center GPU outperforms the CUDA implementation on Nvidia's H100 GPU by a factor up to 2.84 in inference and 1.75 in training. The paper also showcases the efficiency of our SYCL implementation in three significant areas: Image Compression, Neural Radiance Fields, and Physics-Informed Machine Learning. In all cases, our implementation outperforms the off-the-shelf Intel Extension for PyTorch (IPEX) implementation on the same Intel GPU by up to a factor of 30 and the CUDA PyTorch version on Nvidia's H100 GPU by up to a factor 19. The code can be found at https://github.com/intel/tiny-dpcpp-nn.

Figures (7)

• Figure 1: Taxonomy categorizing various applications of MLP. Each category has multiple subcategories that illustrate the specific tasks and domains that MLP can address.
• Figure 2: Illustration of our implementation. On the left hand-side, all multiple layers are sketched. Each layer is parallelized along the batch size (i.e., the $M$ dimension). The right hand-side sketches a single layer. The data of a single sub-group is colored. Green indicates data in the register file. Blue indicates data in the SLM. Each sub-group performs a matrix-matrix multiplication of size $(\mathrm{TM}\times\mathrm{K}) \times (\mathrm{K}\times\mathrm{N})$ utilizing joint_matrix objects of size $\mathrm{TM}\times \mathrm{TK}$, $\mathrm{TK}\times \mathrm{TN}$, and $\mathrm{TM}\times \mathrm{TN}$.
• Figure 3: Comparison of the theoretical peak performance based on the the roofline analysis in Section \ref{['sec:roofline']}.
• Figure 4: Performance of the inference (top) and training (bottom) of our SYCL implementation on a single tile of the Intel Data Center GPU Max 1550 (dark blue) and two tiles of the same Intel GPU (light blue) compared to the CUDA code on a Nvidia H100 GPU (green) for $\mathrm{nlayers} = 6$. The occupancy of the SYCL code on the two tiles of the Intel Data Center GPU is given as the blue line. The x-axis denotes the batch size (i.e., $M$) from $2^{11}$ inputs to $2^{22}$.
• Figure 5: Training progress of Image Compression. The training converges after 1000 steps. The training and inference process is performed for both the SYCL and CUDA implementation and the visualised progress if implemented as per tiny-cuda-nn.