Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Heterogeneous Computing Using SmartNICs and Emerging Data Processing Units

Nathan Tibbetts, Sifat Ibtisum, Satish Puri

TL;DR

This survey analyzes heterogeneous computing with Data Processing Units (DPUs) and SmartNICs, detailing motivations, architectures, and tooling that enable off-path processing and in-network acceleration. It catalogs market DPUs, programming models, and benchmarks, then maps applications across commercial infrastructure, HPC, and AI/ML domains. A central finding is that the largest performance gains come from leveraging onboard hardware accelerators (ASICs/FPGA), while software ecosystems require standardization and cross-vendor collaboration to realize broader benefits. The paper advocates for a cohesive, interoperable suite of DPUs and tooling to unlock widespread adoption and impact in data centers and beyond.

Abstract

The emergence of new, off-path smart network cards (SmartNICs), known generally as Data Processing Units (DPU), has opened a wide range of research opportunities. Of particular interest is the use of these and related devices in tandem with their host's CPU, creating a heterogeneous computing system with new properties and strengths to be explored, capable of accelerating a wide variety of workloads. This survey begins by providing the motivation and relevant background information for this new field, including its origins, a few current hardware offerings, major programming languages and frameworks for using them, and associated challenges. We then review and categorize a number of recent works in the field, covering a wide variety of studies, benchmarks, and application areas, such as data center infrastructure, commercial uses, and AI and ML acceleration. We conclude with a few observations.

A Survey on Heterogeneous Computing Using SmartNICs and Emerging Data Processing Units

TL;DR

This survey analyzes heterogeneous computing with Data Processing Units (DPUs) and SmartNICs, detailing motivations, architectures, and tooling that enable off-path processing and in-network acceleration. It catalogs market DPUs, programming models, and benchmarks, then maps applications across commercial infrastructure, HPC, and AI/ML domains. A central finding is that the largest performance gains come from leveraging onboard hardware accelerators (ASICs/FPGA), while software ecosystems require standardization and cross-vendor collaboration to realize broader benefits. The paper advocates for a cohesive, interoperable suite of DPUs and tooling to unlock widespread adoption and impact in data centers and beyond.

Abstract

The emergence of new, off-path smart network cards (SmartNICs), known generally as Data Processing Units (DPU), has opened a wide range of research opportunities. Of particular interest is the use of these and related devices in tandem with their host's CPU, creating a heterogeneous computing system with new properties and strengths to be explored, capable of accelerating a wide variety of workloads. This survey begins by providing the motivation and relevant background information for this new field, including its origins, a few current hardware offerings, major programming languages and frameworks for using them, and associated challenges. We then review and categorize a number of recent works in the field, covering a wide variety of studies, benchmarks, and application areas, such as data center infrastructure, commercial uses, and AI and ML acceleration. We conclude with a few observations.

Paper Structure

This paper contains 19 sections, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. The Purpose of a DPU
  4. Evolution of NICs and DPUs
  5. Computational Acceleration and Offload
  6. Data Path
  7. Isolation
  8. Challenges
  9. DPUs on the Market
  10. Programming Languages and Frameworks
  11. Creating New Tools
  12. Parallel and Distributed Computing on DPUs
  13. Broad Studies and Benchmarking of DPUs
  14. Application Areas of DPUs
  15. DPUs in Commercial and Infrastructure Applications
  16. ...and 4 more sections

Figures (2)

  • Figure 1: Part of the 2021 ServeTheHome NIC Continuum, a useful classification of existing NIC hardware, and which doubles as a representation of its evolution over time 21thsmartNIC.
  • Figure 2: Illustration of offloading work to DPU hardware within a compute node. Offload methods can take advantage of a variety of tools, such as Direct Memory Access (DMA), illustrated here.