Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploration of Unary Arithmetic-Based Matrix Multiply Units for Low Precision DL Accelerators

Prabhu Vellaisamy, Harideep Nair, Di Wu, Shawn Blanton, John Paul Shen

TL;DR

This work assesses unary GEMM designs (uGEMM, tuGEMM, tubGEMM) against traditional binary GEMM for INT-based DL inference, conducting post-synthesis PPA evaluations across multiple bit-widths and matrix sizes followed by sparsity profiling on CNNs and LLaMA2-70B. It finds that temporal-unary designs (tuGEMM and tubGEMM) offer strong area-power and energy performance, with tubGEMM particularly advantageous when sparsity is present, while binary GEMM retains the best latency-ADP trade-offs. The results illuminate trade-offs among area, power, latency, and sparsity exploitation, suggesting temporal-unary compute as a viable, energy-efficient path for edge AI accelerators in low-precision regimes (2–4 bits). The study thus informs design directions for next-generation DLA architectures prioritizing energy efficiency and sparsity-aware operation at ultra-low precision.

Abstract

General matrix multiplication (GEMM) is a fundamental operation in deep learning (DL). With DL moving increasingly toward low precision, recent works have proposed novel unary GEMM designs as an alternative to conventional binary GEMM hardware. A rigorous evaluation of recent unary and binary GEMM designs is needed to assess the potential of unary hardware for future DL compute. This paper focuses on unary GEMM designs for integer-based DL inference and performs a detailed evaluation of three latest unary design proposals, namely, uGEMM, tuGEMM and tubGEMM, by comparing them to a conventional binary GEMM. Rigorous post-synthesis evaluations beyond prior works are performed across varying bit-widths and matrix sizes to assess the designs' tradeoffs and determine optimal sweetspots. Further, we perform weight sparsity analysis across eight pretrained convolutional neural networks (CNNs) and the LLaMA2 large language model (LLM). In this work, we demonstrate how unary GEMM can be effectively used for energy-efficient compute in future edge AI accelerators.

Exploration of Unary Arithmetic-Based Matrix Multiply Units for Low Precision DL Accelerators

TL;DR

This work assesses unary GEMM designs (uGEMM, tuGEMM, tubGEMM) against traditional binary GEMM for INT-based DL inference, conducting post-synthesis PPA evaluations across multiple bit-widths and matrix sizes followed by sparsity profiling on CNNs and LLaMA2-70B. It finds that temporal-unary designs (tuGEMM and tubGEMM) offer strong area-power and energy performance, with tubGEMM particularly advantageous when sparsity is present, while binary GEMM retains the best latency-ADP trade-offs. The results illuminate trade-offs among area, power, latency, and sparsity exploitation, suggesting temporal-unary compute as a viable, energy-efficient path for edge AI accelerators in low-precision regimes (2–4 bits). The study thus informs design directions for next-generation DLA architectures prioritizing energy efficiency and sparsity-aware operation at ultra-low precision.

Abstract

General matrix multiplication (GEMM) is a fundamental operation in deep learning (DL). With DL moving increasingly toward low precision, recent works have proposed novel unary GEMM designs as an alternative to conventional binary GEMM hardware. A rigorous evaluation of recent unary and binary GEMM designs is needed to assess the potential of unary hardware for future DL compute. This paper focuses on unary GEMM designs for integer-based DL inference and performs a detailed evaluation of three latest unary design proposals, namely, uGEMM, tuGEMM and tubGEMM, by comparing them to a conventional binary GEMM. Rigorous post-synthesis evaluations beyond prior works are performed across varying bit-widths and matrix sizes to assess the designs' tradeoffs and determine optimal sweetspots. Further, we perform weight sparsity analysis across eight pretrained convolutional neural networks (CNNs) and the LLaMA2 large language model (LLM). In this work, we demonstrate how unary GEMM can be effectively used for energy-efficient compute in future edge AI accelerators.
Paper Structure (17 sections, 1 equation, 3 figures, 5 tables)

This paper contains 17 sections, 1 equation, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background on Unary GEMM Architectures
  3. uGEMM
  4. tuGEMM
  5. tubGEMM
  6. Evaluation Methodology
  7. Setup for PPA Analysis
  8. Setup for Weight Sparsity Analysis
  9. Experimental Results
  10. Area-Power Evaluation
  11. Energy Evaluation with Worst-Case Latency
  12. Area-Delay Product Evaluation with Worst-Case Latency
  13. Sparsity-Driven Latency-Energy Evaluation
  14. Potential for Unary-based AI Compute
  15. Temporal-Unary Compute
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Four GEMM designs: uGEMM encodes both inputs as unary (rate/temporal), tuGEMM encodes both inputs as temporal-unary, tubGEMM encodes one input as temporal-unary and other binary, and the conventional binary GEMM. The vertical ordering of the four designs reflects a spectrum from fully-unary design (green) to a fully-binary design (red).
  • Figure 2: 45nm post-synthesis area and power scaling across 2, 4, and 8-bits for 32x32 matrix size. Y-axis is in log scale.
  • Figure 3: Energy consumption for 32x32 GEMM across 8-, 4- and 2-bits. Y-axis is in log scale. Compared to left worst-case plot, note for tubGEMM, increased energy efficiency at 2 bits, earlier cross-over point with bGEMM, and larger energy gap to uGEMM at 8 bits, with sparsity (right plot).