Spiking Neural Network Architecture Search: A Survey

TL;DR

This survey addresses the challenge of designing efficient Spiking Neural Network architectures under hardware constraints by framing Spiking Neural Network Architecture Search (SNNaS) as a hardware/software co-design problem. It surveys fundamental SNN concepts, distinguishes them from traditional ANNs, and analyzes how Neural Architecture Search must be adapted for spike-based computation, focusing on search spaces, strategies, evaluation, and acceleration. Key contributions include a taxonomy of search spaces (global, sequential, cell-based, hierarchical, memory-bank), hardware-aware NAS, and co-exploration approaches (e.g., ANAS, ANCoEF) that demonstrate substantial improvements in energy-delay and search efficiency. The paper highlights practical implications for neuromorphic hardware deployment, underscores open challenges in scalability and reproducibility, and outlines future directions such as surrogate-gradient optimization, Bayesian methods, and deeper hardware-software integration to unlock the full potential of SNNaS in edge AI and real-time applications.

Abstract

This survey paper presents a comprehensive examination of Spiking Neural Network (SNN) architecture search (SNNaS) from a unique hardware/software co-design perspective. SNNs, inspired by biological neurons, have emerged as a promising approach to neuromorphic computing. They offer significant advantages in terms of power efficiency and real-time resource-constrained processing, making them ideal for edge computing and IoT applications. However, designing optimal SNN architectures poses significant challenges, due to their inherent complexity (e.g., with respect to training) and the interplay between hardware constraints and SNN models. We begin by providing an overview of SNNs, emphasizing their operational principles and key distinctions from traditional artificial neural networks (ANNs). We then provide a brief overview of the state of the art in NAS for ANNs, highlighting the challenges of directly applying these approaches to SNNs. We then survey the state-of-the-art in SNN-specific NAS approaches. Finally, we conclude with insights into future research directions for SNN research, emphasizing the potential of hardware/software co-design in unlocking the full capabilities of SNNs. This survey aims to serve as a valuable resource for researchers and practitioners in the field, offering a holistic view of SNNaS and underscoring the importance of a co-design approach to harness the true potential of neuromorphic computing.

Figures (9)

• Figure 1: Overview of the Neural Architecture Search (NAS) workflow for Spiking Neural Networks: From defining the search space (e.g., layer configurations, neuron models, and hyperparameters) to employing search strategies (e.g., evolutionary algorithms, gradient-based search, and reinforcement learning, etc.) for performance evaluation (e.g., accuracy, latency, and energy) and, finally, optimal architecture selection.
• Figure 2: Neural Architecture Search has three important components: search space, search strategy, and evaluation elsken2019neural.
• Figure 3: Example of a cell with four nodes and five possible operations. The cell is a directed acyclic graph, with each edge representing an operation dong2020bench.
• Figure 4: Example of a three-level hierarchy taken from Liu et al. liu2017hierarchical. Level-1 primitive operations $o_1^{(1)}$, $o_2^{(1)}$, and $o_3^{(1)}$, representing $1 \times 1$ convolution, $3 \times 3$ convolution, and $3 \times 3$ max-pooling respectively, are assembled into a level-2 motif, $o_1^{(2)}$ (TOP). Level-2 motifs $o_1^{(2)}$, $o_2^{(2)}$, and $o_3^{(2)}$ are then assembled into a level-3 motif $o_1^{(3)}$ (BOTTOM).
• Figure 5: Example of a memory-bank representation taken from Brock et al. brock2017smash. In a single-branch architecture (LEFT), the network utilizes a singular, large memory bank that it both reads and overwrites at each operation. Branching architectures (RIGHT) read from all previously written banks and write to an empty bank.