Evaluating Emerging AI/ML Accelerators: IPU, RDU, and NVIDIA/AMD GPUs

TL;DR

A preliminary evaluation and comparison of commercial AI/ML accelerators, delving into their hardware and software design features to discern their strengths and unique capabilities and aims to illuminate the advantages of data-flow architectures over conventional processor designs and offer insights into the performance trade-offs of each platform.

Abstract

The relentless advancement of artificial intelligence (AI) and machine learning (ML) applications necessitates the development of specialized hardware accelerators capable of handling the increasing complexity and computational demands. Traditional computing architectures, based on the von Neumann model, are being outstripped by the requirements of contemporary AI/ML algorithms, leading to a surge in the creation of accelerators like the Graphcore Intelligence Processing Unit (IPU), Sambanova Reconfigurable Dataflow Unit (RDU), and enhanced GPU platforms. These hardware accelerators are characterized by their innovative data-flow architectures and other design optimizations that promise to deliver superior performance and energy efficiency for AI/ML tasks. This research provides a preliminary evaluation and comparison of these commercial AI/ML accelerators, delving into their hardware and software design features to discern their strengths and unique capabilities. By conducting a series of benchmark evaluations on common DNN operators and other AI/ML workloads, we aim to illuminate the advantages of data-flow architectures over conventional processor designs and offer insights into the performance trade-offs of each platform. The findings from our study will serve as a valuable reference for the design and performance expectations of research prototypes, thereby facilitating the development of next-generation hardware accelerators tailored for the ever-evolving landscape of AI/ML applications. Through this analysis, we aspire to contribute to the broader understanding of current accelerator technologies and to provide guidance for future innovations in the field.

Figures (6)

• Figure 1: (a) Graphcore IPU architecture. (b) Sambanova RDU architecture
• Figure 2: Cross platform evaluation on square GEMM operators.
• Figure 3: Cross platform evaluation on BERT operators.
• Figure 4: Cross platform evaluation on 2D convolution operators.
• Figure 5: Cross platform evaluation on SPMM operators.