Modeling and Optimizing Performance Bottlenecks for Neuromorphic Accelerators

TL;DR

This work presents a bound-and-bottleneck analysis for neuromorphic accelerators, arguing that neurocore-level load balance, rather than network-wide sparsity, governs deployed performance. It introduces the floorline performance model to visualize memory, compute, and traffic bottlenecks and proposes a two-stage optimization combining sparsity-aware training with floorline-informed partitioning. Across three real accelerators (AKD1000, Speck, Loihi 2), the authors demonstrate three bottleneck regimes—memory-bound, compute-bound, and traffic-bound—and validate that floorline-guided optimization yields substantial iso-accuracy gains: up to 3.86× runtime and 3.38× energy reductions. The study provides a principled framework for predicting neuromorphic performance and guiding practical workload tuning, with a pathway toward broader hardware support and multi-chip extensions.

Abstract

Neuromorphic accelerators offer promising platforms for machine learning (ML) inference by leveraging event-driven, spatially-expanded architectures that naturally exploit unstructured sparsity through co-located memory and compute. However, their unique architectural characteristics create performance dynamics that differ fundamentally from conventional accelerators. Existing workload optimization approaches for neuromorphic accelerators rely on aggregate network-wide sparsity and operation counting, but the extent to which these metrics actually improve deployed performance remains unknown. This paper presents the first comprehensive performance bound and bottleneck analysis of neuromorphic accelerators, revealing the shortcomings of the conventional metrics and offering an understanding of what facets matter for workload performance. We present both theoretical analytical modeling and extensive empirical characterization of three real neuromorphic accelerators: Brainchip AKD1000, Synsense Speck, and Intel Loihi 2. From these, we establish three distinct accelerator bottleneck states, memory-bound, compute-bound, and traffic-bound, and identify which workload configuration features are likely to exhibit these bottleneck states. We synthesize all of our insights into the floorline performance model, a visual model that identifies performance bounds and informs how to optimize a given workload, based on its position on the model. Finally, we present an optimization methodology that combines sparsity-aware training with floorline-informed partitioning. Our methodology achieves substantial performance improvements at iso-accuracy: up to 3.86x runtime improvement and 3.38x energy reduction compared to prior manually-tuned configurations.

Figures (12)

• Figure 1: The general macro-architecture and compute flow of neural network execution on neuromorphic accelerators. Blue components denote co-located memory, orange denotes compute, and red denotes inter-core communication via the NoC.
• Figure 2: Weight sparsity performance of CNNs.
• Figure 3: S5 (Loihi 2) weight sparsity performance
• Figure 4: Timing overhead of sparse weight support on Loihi 2. Solid lines use dense weight formatting, while dashed lines use sparse weight formatting.
• Figure 5: Activation sparsity with varying sparsity schedules to change load balance