A Review on Proprietary Accelerators for Large Language Models

TL;DR

The paper addresses the escalating hardware demands of large language models by surveying proprietary accelerator technologies implemented as ASICs and analyzing their hardware and software ecosystems. It offers a structured comparison of major commercial accelerators (e.g., H100, MI300X, WSE-2, TPU v4, IPU, LPU, Gaudi 3, SN40L, Grayskull) across memory architectures, interconnects, compute throughput, and software support to illuminate current tradeoffs. Key contributions include identifying persistent challenges—memory capacity and cost, power consumption, scalability, and software/compiler compatibility—and outlining directions for future ASIC designs and optimization of software toolchains. The findings have practical relevance for data-center deployments and guide research toward energy-efficient, scalable LLM acceleration with robust framework support.

Abstract

With the advancement of Large Language Models (LLMs), the importance of accelerators that efficiently process LLM computations has been increasing. This paper discusses the necessity of LLM accelerators and provides a comprehensive analysis of the hardware and software characteristics of the main commercial LLM accelerators. Based on this analysis, we propose considerations for the development of next-generation LLM accelerators and suggest future research directions.