Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lincoln AI Computing Survey (LAICS) and Trends

Albert Reuther, Peter Michaleas, Michael Jones, Vijay Gadepally, Jeremy Kepner

TL;DR

The paper updates the Lincoln AI Computing Survey (LAICS) by compiling and analyzing publicly announced AI accelerators across edge to data-center deployments, focusing on peak performance and peak power to map inference- and training-oriented devices. It leverages scatter plots with market-segment zooms and introduces a revised architectural categorization to compare diverse compute substrates, highlighting trends in precision usage and architectural choices. The study documents Nvidia’s ongoing data-center dominance while detailing rising competition from AMD, Groq, Cerebras, and several new entrants, as well as ongoing exploration of non-CMOS approaches such as optical computing. The work provides a transparent, data-driven snapshot of the AI accelerator landscape and establishes a framework for tracking future releases through publicly available data on GitHub.

Abstract

In the past year, generative AI (GenAI) models have received a tremendous amount of attention, which in turn has increased attention to computing systems for training and inference for GenAI. Hence, an update to this survey is due. This paper is an update of the survey of AI accelerators and processors from past seven years, which is called the Lincoln AI Computing Survey -- LAICS (pronounced "lace"). This multi-year survey collects and summarizes the current commercial accelerators that have been publicly announced with peak performance and peak power consumption numbers. In the same tradition of past papers of this survey, the performance and power values are plotted on a scatter graph, and a number of dimensions and observations from the trends on this plot are again discussed and analyzed. Market segments are highlighted on the scatter plot, and zoomed plots of each segment are also included. A brief description of each of the new accelerators that have been added in the survey this year is included, and this update features a new categorization of computing architectures that implement each of the accelerators.

Lincoln AI Computing Survey (LAICS) and Trends

TL;DR

The paper updates the Lincoln AI Computing Survey (LAICS) by compiling and analyzing publicly announced AI accelerators across edge to data-center deployments, focusing on peak performance and peak power to map inference- and training-oriented devices. It leverages scatter plots with market-segment zooms and introduces a revised architectural categorization to compare diverse compute substrates, highlighting trends in precision usage and architectural choices. The study documents Nvidia’s ongoing data-center dominance while detailing rising competition from AMD, Groq, Cerebras, and several new entrants, as well as ongoing exploration of non-CMOS approaches such as optical computing. The work provides a transparent, data-driven snapshot of the AI accelerator landscape and establishes a framework for tracking future releases through publicly available data on GitHub.

Abstract

In the past year, generative AI (GenAI) models have received a tremendous amount of attention, which in turn has increased attention to computing systems for training and inference for GenAI. Hence, an update to this survey is due. This paper is an update of the survey of AI accelerators and processors from past seven years, which is called the Lincoln AI Computing Survey -- LAICS (pronounced "lace"). This multi-year survey collects and summarizes the current commercial accelerators that have been publicly announced with peak performance and peak power consumption numbers. In the same tradition of past papers of this survey, the performance and power values are plotted on a scatter graph, and a number of dimensions and observations from the trends on this plot are again discussed and analyzed. Market segments are highlighted on the scatter plot, and zoomed plots of each segment are also included. A brief description of each of the new accelerators that have been added in the survey this year is included, and this update features a new categorization of computing architectures that implement each of the accelerators.
Paper Structure (7 sections, 3 figures)

This paper contains 7 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Survey of Processors
  4. Observations and Trends
  5. Non-CMOS Technologies
  6. Summary
  7. Data Availability

Figures (3)

  • Figure 1: Peak performance vs. power scatter plot of publicly announced AI accelerators and processors.
  • Figure 2: Zoomed regions of peak performance vs. peak power scatter plot: (a) very low power, (b) embedded, (c) autonomous, (d) data center chips and cards, (e) data center systems.
  • Figure 3: Range of AI accelerator computer architecture categories. Going from left to right, greater optimization of data movement between computations means data travels less distances between computations, and more computations executed in parallel. However, it also means less flexibility in operation types and programmability. CPU = Central Processing Unit; AVX = Advanced Vector eXtensions; SVE = Scalable Vector Extensions; GPU = Graphics Processing Unit; TPU = Tensor Processing Unit; CGRA = Course Grained Reconfigurable Architecture FPGA = Field Programmable Gate Array; ASIC = Application Specific Integrated Circuit