Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SpikeX: Exploring Accelerator Architecture and Network-Hardware Co-Optimization for Sparse Spiking Neural Networks

Boxun Xu, Richard Boone, Peng Li

TL;DR

SpikeX presents a novel sparse-friendly SNN accelerator built on a systolic array that actively exploits unstructured spatiotemporal sparsity through Agile SpatioTemporal Dispatch and Activation-induced Weight Tailoring. The authors couple hardware-aware training (SpikeX-HT) with hardware architecture search (SpikeX-HAS) to co-optimize networks and accelerator configurations, achieving substantial reductions in energy and latency while maintaining accuracy. Key contributions include a three-level weight reuse strategy, a detailed memory hierarchy and NTWU-based workload packing, and a formalized co-optimization framework that links sparsity to energy-delay performance. The results demonstrate broad applicability across DVS-Gesture and NMNIST benchmarks, delivering up to $150.87\times$ EDP improvement over prior baselines, with robust generalization to different SNN styles.

Abstract

Spiking Neural Networks (SNNs) are promising biologically plausible models of computation which utilize a spiking binary activation function similar to that of biological neurons. SNNs are well positioned to process spatiotemporal data, and are advantageous in ultra-low power and real-time processing. Despite a large body of work on conventional artificial neural network accelerators, much less attention has been given to efficient SNN hardware accelerator design. In particular, SNNs exhibit inherent unstructured spatial and temporal firing sparsity, an opportunity yet to be fully explored for great hardware processing efficiency. In this work, we propose a novel systolic-array SNN accelerator architecture, called SpikeX, to take on the challenges and opportunities stemming from unstructured sparsity while taking into account the unique characteristics of spike-based computation. By developing an efficient dataflow targeting expensive multi-bit weight data movements, SpikeX reduces memory access and increases data sharing and hardware utilization for computations spanning across both time and space, thereby significantly improving energy efficiency and inference latency. Furthermore, recognizing the importance of SNN network and hardware co-design, we develop a co-optimization methodology facilitating not only hardware-aware SNN training but also hardware accelerator architecture search, allowing joint network weight parameter optimization and accelerator architectural reconfiguration. This end-to-end network/accelerator co-design approach offers a significant reduction of 15.1x-150.87x in energy-delay-product(EDP) without comprising model accuracy.

SpikeX: Exploring Accelerator Architecture and Network-Hardware Co-Optimization for Sparse Spiking Neural Networks

TL;DR

SpikeX presents a novel sparse-friendly SNN accelerator built on a systolic array that actively exploits unstructured spatiotemporal sparsity through Agile SpatioTemporal Dispatch and Activation-induced Weight Tailoring. The authors couple hardware-aware training (SpikeX-HT) with hardware architecture search (SpikeX-HAS) to co-optimize networks and accelerator configurations, achieving substantial reductions in energy and latency while maintaining accuracy. Key contributions include a three-level weight reuse strategy, a detailed memory hierarchy and NTWU-based workload packing, and a formalized co-optimization framework that links sparsity to energy-delay performance. The results demonstrate broad applicability across DVS-Gesture and NMNIST benchmarks, delivering up to EDP improvement over prior baselines, with robust generalization to different SNN styles.

Abstract

Spiking Neural Networks (SNNs) are promising biologically plausible models of computation which utilize a spiking binary activation function similar to that of biological neurons. SNNs are well positioned to process spatiotemporal data, and are advantageous in ultra-low power and real-time processing. Despite a large body of work on conventional artificial neural network accelerators, much less attention has been given to efficient SNN hardware accelerator design. In particular, SNNs exhibit inherent unstructured spatial and temporal firing sparsity, an opportunity yet to be fully explored for great hardware processing efficiency. In this work, we propose a novel systolic-array SNN accelerator architecture, called SpikeX, to take on the challenges and opportunities stemming from unstructured sparsity while taking into account the unique characteristics of spike-based computation. By developing an efficient dataflow targeting expensive multi-bit weight data movements, SpikeX reduces memory access and increases data sharing and hardware utilization for computations spanning across both time and space, thereby significantly improving energy efficiency and inference latency. Furthermore, recognizing the importance of SNN network and hardware co-design, we develop a co-optimization methodology facilitating not only hardware-aware SNN training but also hardware accelerator architecture search, allowing joint network weight parameter optimization and accelerator architectural reconfiguration. This end-to-end network/accelerator co-design approach offers a significant reduction of 15.1x-150.87x in energy-delay-product(EDP) without comprising model accuracy.
Paper Structure (27 sections, 5 equations, 13 figures, 2 tables)

This paper contains 27 sections, 5 equations, 13 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Existing Work
  3. Existing SNN Hardware Accelerators
  4. Challenges and Opportunities in SNN Hardware Acceleration
  5. SpikeX Architecture
  6. Overview of the SpikeX Architecture
  7. Agile SpatioTemporal Dispatch
  8. Neuro-temporal Work Units (NTWU)
  9. Agile NTWU Dispatch with Three-Level of Weight Reuse
  10. Activation-induced Weight Tailoring
  11. Network and Accelerator Co-Optimization for Spiking Neural Networks(SNNs)
  12. Hardware-Aware Training (SpikeX-HT)
  13. Hardware Architecture Search (SpikeX-HAS)
  14. Evaluation Methodology
  15. Modeling of systolic array and Hierarchical Memory
  16. ...and 12 more sections

Figures (13)

  • Figure 1: (a) The proposed SpikeX hardware Architecture (b) The schematic design of Processing Element (PE) in the systolic array (c) A comprehensive weight sharing scheme that operates at three levels: (1) within the same time window (TW), (2) across multiple TWs, and (3) across post-synaptic neurons
  • Figure 2: Four temporal granularities: time point, time window (TW), time block (TB), and time stride(TS), and the corresponding activity tags.
  • Figure 3: Agile SpatioTemporal Dispatch. (Left) High temporal density mode, where NTWUs across TW grids are dispatched onto distinct PE columns. (Right) High spatial density mode, where NTWUs across various spatial grids and TW grids are dispatched onto different PE columns.
  • Figure 4: Activation-induced weight tailoring during determining necessary weight kernels loaded to computation units.
  • Figure 5: An analysis of realistic layer-wise firing activities of LIF neurons and structured sparsity of packed spiking time windows(TWs) with and without SpikeX-HT, on DVSG medium, DVSG large and NMNIST.
  • ...and 8 more figures