Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Unified Programming Model for Heterogeneous Computing with CPU and Accelerator Technologies

Yuqing Xiong

TL;DR

Heterogeneous computing across CPUs and accelerators introduces programming and data-movement challenges. The paper proposes the Unified Programming Model (UPM), which treats accelerators as I/O devices and extends file management to cover them, enabling cross-device programming with standard languages and reducing all architectures to a Type-I base. It formalizes a taxonomy of computer-system types and shows reductions to Type-I, while also enabling coupled applications computing via MPI through a unified API. The approach aims to improve software productivity, simplify cross-architecture development, and provide a natural pathway for multidisciplinary simulations across diverse platforms.

Abstract

This paper consists of three parts. The first part provides a unified programming model for heterogeneous computing with CPU and accelerator (like GPU, FPGA, Google TPU, Atos QPU, and more) technologies. To some extent, this new programming model makes programming across CPUs and accelerators turn into usual programming tasks with common programming languages, and relieves complexity of programming across CPUs and accelerators. It can be achieved by extending file managements in common programming languages, such as C/C++, Fortran, Python, MPI, etc., to cover accelerators as I/O devices. In the second part, we show that all types of computer systems can be reduced to the simplest type of computer system, a single-core CPU computer system with I/O devices, by the unified programming model. Thereby, the unified programming model can truly build the programming of various computer systems on one API (i.e. file managements of common programming languages), and can make programming for various computer systems easier. In third part, we present a new approach to coupled applications computing (like multidisciplinary simulations) by the unified programming model. The unified programming model makes coupled applications computing more natural and easier since it only relies on its own power to couple multiple applications through MPI.

A Unified Programming Model for Heterogeneous Computing with CPU and Accelerator Technologies

TL;DR

Heterogeneous computing across CPUs and accelerators introduces programming and data-movement challenges. The paper proposes the Unified Programming Model (UPM), which treats accelerators as I/O devices and extends file management to cover them, enabling cross-device programming with standard languages and reducing all architectures to a Type-I base. It formalizes a taxonomy of computer-system types and shows reductions to Type-I, while also enabling coupled applications computing via MPI through a unified API. The approach aims to improve software productivity, simplify cross-architecture development, and provide a natural pathway for multidisciplinary simulations across diverse platforms.

Abstract

This paper consists of three parts. The first part provides a unified programming model for heterogeneous computing with CPU and accelerator (like GPU, FPGA, Google TPU, Atos QPU, and more) technologies. To some extent, this new programming model makes programming across CPUs and accelerators turn into usual programming tasks with common programming languages, and relieves complexity of programming across CPUs and accelerators. It can be achieved by extending file managements in common programming languages, such as C/C++, Fortran, Python, MPI, etc., to cover accelerators as I/O devices. In the second part, we show that all types of computer systems can be reduced to the simplest type of computer system, a single-core CPU computer system with I/O devices, by the unified programming model. Thereby, the unified programming model can truly build the programming of various computer systems on one API (i.e. file managements of common programming languages), and can make programming for various computer systems easier. In third part, we present a new approach to coupled applications computing (like multidisciplinary simulations) by the unified programming model. The unified programming model makes coupled applications computing more natural and easier since it only relies on its own power to couple multiple applications through MPI.
Paper Structure (19 sections, 6 figures)

This paper contains 19 sections, 6 figures.

Table of Contents

  1. Introduction
  2. A Unified Programming Model for Heterogeneous Computing with CPU and Accelerator Technologies
  3. Accelerators as One Kind of I/O Devices in Common Programming Languages
  4. For Multi-core CPUs for OpenMP or Pthread in UPM
  5. For Clusters of CPUs for MPI in UPM
  6. Computer Systems under the View of UPM
  7. All Types of Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  8. Type-I Computer Systems
  9. Type-$\rm I^+$ Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  10. Type-II Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  11. Type-$\rm II^+$ Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  12. Type-III Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  13. Type-$\rm III^+$ Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  14. Type-IV Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  15. Type-$\rm IV^+$ Computer Systems $\stackrel{UPM}\Longrightarrow$ Type-I Computer Systems
  16. ...and 4 more sections

Figures (6)

  • Figure 1: GPU Scale out using Vega20 and Epyc (extracted from [1])
  • Figure 2: FPGA Exploitation (extracted from [1])
  • Figure 3: Communication between P running on a CPU and Q running on a GPU.
  • Figure 4: A coupled applications computing.
  • Figure 5: The clusters as general printers in the coupled applications computing by UPM.
  • ...and 1 more figures