Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Photonic-digital Accelerator for Attention Mechanism

Huize Li, Dan Chen, Tulika Mitra

TL;DR

HyAtten tackles the high signal-conversion cost in photonic Transformer accelerators by classifying analog signals into low-resolution (≤4-bit) and high-resolution (>4-bit) streams. It uses a photonic die with many low-bit ADCs to process the majority of signals and a digital die to handle the minority high-resolution signals, reducing area, latency, and energy overhead. The architecture employs Tile-level DPTC-based GEMMs, an analog comparator to route oversize signals to digital paths, and a softmax unit for attention, achieving up to 9.8× speedup per unit area and 2.2× energy efficiency per unit area over a photonic baseline, while maintaining negligible accuracy loss. This hybrid approach enables scalable, energy-efficient attention computation and demonstrates the practical viability of photonic accelerators for large-scale Transformers.

Abstract

The wide adoption and substantial computational resource requirements of attention-based Transformers have spurred the demand for efficient hardware accelerators. Unlike digital-based accelerators, there is growing interest in exploring photonics due to its high energy efficiency and ultra-fast processing speeds. However, the significant signal conversion overhead limits the performance of photonic-based accelerators. In this work, we propose HyAtten, a photonic-based attention accelerator with minimize signal conversion overhead. HyAtten incorporates a signal comparator to classify signals into two categories based on whether they can be processed by low-resolution converters. HyAtten integrates low-resolution converters to process all low-resolution signals, thereby boosting the parallelism of photonic computing. For signals requiring high-resolution conversion, HyAtten uses digital circuits instead of signal converters to reduce area and latency overhead. Compared to state-of-the-art photonic-based Transformer accelerator, HyAtten achieves 9.8X performance/area and 2.2X energy-efficiency/area improvement.

Hybrid Photonic-digital Accelerator for Attention Mechanism

TL;DR

HyAtten tackles the high signal-conversion cost in photonic Transformer accelerators by classifying analog signals into low-resolution (≤4-bit) and high-resolution (>4-bit) streams. It uses a photonic die with many low-bit ADCs to process the majority of signals and a digital die to handle the minority high-resolution signals, reducing area, latency, and energy overhead. The architecture employs Tile-level DPTC-based GEMMs, an analog comparator to route oversize signals to digital paths, and a softmax unit for attention, achieving up to 9.8× speedup per unit area and 2.2× energy efficiency per unit area over a photonic baseline, while maintaining negligible accuracy loss. This hybrid approach enables scalable, energy-efficient attention computation and demonstrates the practical viability of photonic accelerators for large-scale Transformers.

Abstract

The wide adoption and substantial computational resource requirements of attention-based Transformers have spurred the demand for efficient hardware accelerators. Unlike digital-based accelerators, there is growing interest in exploring photonics due to its high energy efficiency and ultra-fast processing speeds. However, the significant signal conversion overhead limits the performance of photonic-based accelerators. In this work, we propose HyAtten, a photonic-based attention accelerator with minimize signal conversion overhead. HyAtten incorporates a signal comparator to classify signals into two categories based on whether they can be processed by low-resolution converters. HyAtten integrates low-resolution converters to process all low-resolution signals, thereby boosting the parallelism of photonic computing. For signals requiring high-resolution conversion, HyAtten uses digital circuits instead of signal converters to reduce area and latency overhead. Compared to state-of-the-art photonic-based Transformer accelerator, HyAtten achieves 9.8X performance/area and 2.2X energy-efficiency/area improvement.
Paper Structure (17 sections, 2 equations, 8 figures, 2 tables)

This paper contains 17 sections, 2 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Transformer and self-attention
  4. Optical computing basics
  5. Motivations
  6. HyAtten
  7. Architecture
  8. Dataflow
  9. Experimental Evaluation
  10. Experimental Setup
  11. Compare to State-of-the-art Photonic Accelerator
  12. Compare to State-of-the-art Digital Accelerators
  13. Accuracy Comparison
  14. System Efficiency Analysis
  15. Related Work
  16. ...and 2 more sections

Figures (8)

  • Figure 1: (a) DPTC array proposed in Zhu24, and (b) DDot unit proposed in Zhu24
  • Figure 2: (a) The model accuracy when employing different ADC resolutions, and (b) the proportion of signals that remain within the resolution limits of various ADCs
  • Figure 3: Architecture and dataflow of HyAtten
  • Figure 4: GEMM operations on multiple photonic Tiles
  • Figure 5: (a) Performance per unit area, and (b) energy efficiency per unit area
  • ...and 3 more figures