Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

tuGEMM: Area-Power-Efficient Temporal Unary GEMM Architecture for Low-Precision Edge AI

Harideep Nair, Prabhu Vellaisamy, Albert Chen, Joseph Finn, Anna Li, Manav Trivedi, John Paul Shen

TL;DR

This work addresses the need for area- and power-efficient GEMM on edge devices by replacing stochastic unary approaches with exact temporal unary compute. It introduces tuGEMM, a temporal-coding GEMM architecture available in serial and parallel variants, and reports gate-level implementations with post-synthesis metrics in 45 nm across 2-, 4-, and 8-bit widths. The key contributions are the temporal encoding scheme, the serial and parallel designs with detailed component-level descriptions, and comprehensive PPA and latency evaluations, including accuracy results that favor exact compute (e.g., 96.08% vs 94.7% for a related uGEMM). The results indicate substantial area-power reductions compared to state-of-the-art unary systems, especially at low precision, making tuGEMM well-suited for always-on edge AI and real-time sensing applications, with potential integration into DL accelerators.

Abstract

General matrix multiplication (GEMM) is a ubiquitous computing kernel/algorithm for data processing in diverse applications, including artificial intelligence (AI) and deep learning (DL). Recent shift towards edge computing has inspired GEMM architectures based on unary computing, which are predominantly stochastic and rate-coded systems. This paper proposes a novel GEMM architecture based on temporal-coding, called tuGEMM, that performs exact computation. We introduce two variants of tuGEMM, serial and parallel, with distinct area/power-latency trade-offs. Post-synthesis Power-Performance-Area (PPA) in 45 nm CMOS are reported for 2-bit, 4-bit, and 8-bit computations. The designs illustrate significant advantages in area-power efficiency over state-of-the-art stochastic unary systems especially at low precisions, e.g. incurring just 0.03 mm^2 and 9 mW for 4 bits, and 0.01 mm^2 and 4 mW for 2 bits. This makes tuGEMM ideal for power constrained mobile and edge devices performing always-on real-time sensory processing.

tuGEMM: Area-Power-Efficient Temporal Unary GEMM Architecture for Low-Precision Edge AI

TL;DR

This work addresses the need for area- and power-efficient GEMM on edge devices by replacing stochastic unary approaches with exact temporal unary compute. It introduces tuGEMM, a temporal-coding GEMM architecture available in serial and parallel variants, and reports gate-level implementations with post-synthesis metrics in 45 nm across 2-, 4-, and 8-bit widths. The key contributions are the temporal encoding scheme, the serial and parallel designs with detailed component-level descriptions, and comprehensive PPA and latency evaluations, including accuracy results that favor exact compute (e.g., 96.08% vs 94.7% for a related uGEMM). The results indicate substantial area-power reductions compared to state-of-the-art unary systems, especially at low precision, making tuGEMM well-suited for always-on edge AI and real-time sensing applications, with potential integration into DL accelerators.

Abstract

General matrix multiplication (GEMM) is a ubiquitous computing kernel/algorithm for data processing in diverse applications, including artificial intelligence (AI) and deep learning (DL). Recent shift towards edge computing has inspired GEMM architectures based on unary computing, which are predominantly stochastic and rate-coded systems. This paper proposes a novel GEMM architecture based on temporal-coding, called tuGEMM, that performs exact computation. We introduce two variants of tuGEMM, serial and parallel, with distinct area/power-latency trade-offs. Post-synthesis Power-Performance-Area (PPA) in 45 nm CMOS are reported for 2-bit, 4-bit, and 8-bit computations. The designs illustrate significant advantages in area-power efficiency over state-of-the-art stochastic unary systems especially at low precisions, e.g. incurring just 0.03 mm^2 and 9 mW for 4 bits, and 0.01 mm^2 and 4 mW for 2 bits. This makes tuGEMM ideal for power constrained mobile and edge devices performing always-on real-time sensory processing.

Paper Structure

This paper contains 17 sections, 1 equation, 5 figures, 1 table.

Table of Contents

  1. Introduction and Background
  2. tuGEMM Architecture
  3. Input Encoding
  4. Serial Architecture
  5. index counter
  6. vector generator
  7. column/row counters
  8. output counter array
  9. Parallel Architecture
  10. vector counters
  11. output adder array
  12. Evaluation and Results
  13. Post-Synthesis Area-Power Evaluation
  14. Latency Evaluation
  15. Worst-Case Latency
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Serial tuGEMM architecture for 4x4 GEMM compute
  • Figure 2: Parallel tuGEMM architecture for 4x4 GEMM compute
  • Figure 3: A single output adder cell in parallel tuGEMM
  • Figure 4: PPA comparison for 16x16 GEMM implementations (serial/parallel tuGEMM vs. uGEMM) across 2, 4, 8 bitwidths.
  • Figure 5: Percentage of GEMM operations that involve the corresponding X-axis values as the maximum magnitude during inference of INT8 quantized ResNet18. Right Y-axis plots the cumulative percentage of operations with maximum values less than or equal to the corresponding X-axis value.