A Compute and Communication Runtime Model for Loihi 2

TL;DR

Neuromorphic computation promises high efficiency, but runtime prediction remains challenging due to compute-communication overlap and NoC congestion. The authors propose a max-affine runtime model for Loihi 2 that jointly accounts for SynOps, SynMem reads, DendOps, and NoC traffic, calibrated via a microbenchmark suite. They validate the model against dense linear layer and QUBO solver workloads, achieving Pearson correlations ≥ 0.97 and often near-perfect alignment, and they derive analytical expressions for traffic-driven scaling. The model enables design of fast kernels on Loihi 2 and provides insight into placement strategies and NoC bottlenecks, informing algorithm-hardware co-design for neuromorphic devices.

Abstract

Neuromorphic computers hold the potential to vastly improve the speed and efficiency of a wide range of computational kernels with their asynchronous, compute-memory co-located, spatially distributed, and scalable nature. However, performance models that are simple yet sufficiently expressive to predict runtime on actual neuromorphic hardware are lacking, posing a challenge for researchers and developers who strive to design fast algorithms and kernels. As breaking the memory bandwidth wall of conventional von-Neumann architectures is a primary neuromorphic advantage, modeling communication time is especially important. At the same time, modeling communication time is difficult, as complex congestion patterns arise in a heavily-loaded Network-on-Chip. In this work, we introduce the first max-affine lower-bound runtime model -- a multi-dimensional roofline model -- for Intel's Loihi 2 neuromorphic chip that quantitatively accounts for both compute and communication based on a suite of microbenchmarks. Despite being a lower-bound model, we observe a tight correspondence (Pearson correlation coefficient greater than or equal to 0.97) between our model's estimated runtime and the measured runtime on Loihi 2 for a neural network linear layer, i.e., matrix-vector multiplication, and for an example application, a Quadratic Unconstrained Binary Optimization solver. Furthermore, we derive analytical expressions for communication-bottlenecked runtime to study scalability of the linear layer, revealing an area-runtime tradeoff for different spatial workload configurations with linear to superliner runtime scaling in layer size with a variety of constant factors. Our max-affine runtime model helps empower the design of high-speed algorithms and kernels for Loihi 2.

Figures (8)

• Figure 1: Schematic illustrating the primary components of Loihi 2 (top half of chip shown): NeuroCores, routers, and Network-on-Chip (NoC) mesh links. NeuroCores perform computations, emit integer-valued messages known as graded spikes, and store synaptic connectivity matrices to weigh incoming graded spikes from other NeuroCores. Routers direct messages from one NeuroCore to another along the NoC mesh links according to dimension-order routing: horizontally first, and then vertically. Two example message paths are illustrated with red and blue arrows.
• Figure 2: Microbenchmark suite. (a) Barrier synchronization microbenchmark: dummy network with one sending and receiving neuron and no messages sent. Runtime is dominated by timestep coordination through the barrier synchronization mechanism. (b) DendOp microbenchmark: a single-core workload with many neurons that perform neuron updates (DendOps); DendOps dominate the processing time. (c) SynOp microbenchmark: $N$ neurons send to $N$ neurons through a dense-packed all-ones synaptic weight matrix; a dominating $N^2$ SynOps are performed each timestep. (d) SynMem read microbenchmark: a dense-packed identity synaptic connectivity matrix reduces SynOps to $N$ per timestep, while keeping a dominant $O(N^2)$ SynMem reads per timestep. (e) Link bandwidth microbenchmark: $N$ messages are transferred from one core to another and the sparse-packed identity matrix connectivity results in few SynOps and SynMem reads. This workload is replicated to put heavy traffic along a common link (Figure \ref{['fig:microbenchmarks_2']}a), whose limited bandwidth dominates runtime.
• Figure 3: NoC traffic visualizations. (a) Mesh link bandwidth microbenchmark. Several replicas (in this case 12) of the link bandwidth workload (Figure \ref{['fig:microbenchmarks_1']}e) are arranged in a single router column; this stresses the vertical link, allowing the peak link bandwidth to be measured. (b) and (c) Two example NeuroCore placement patterns. The same 16 NeuroCores of the tiled-identity linear layer workload are mapped to different locations according to a (b) random placement or (c) X-shaped placement. Aggregate traffic emanating from each router is illustrated as a colored rectangle near the router, and the two halves of the rectangle represent the two meshes on Loihi 2; coloring is relative to within each panel; the upper right corner of the X-shape is offset due to the default reservation of the displaced cores for special purposes.
• Figure 4: Microbenchmark measurements. As each microbenchmark is scaled, the time per step asymptotes to a particular value representing peak system performance for the component stressed by the microbenchmark. This allows the implied effective (a) DendOp time, (b) SynOp time, (c) SynMem read time, and (d) link bandwidth to be calculated. †Workloads characterized on an Oheo Gulch system with a N3C2-revision Loihi 2 chip running an unreleased patch of NxCore 2.5.10 and NxKernel 0.4.0.
• Figure 5: Correspondence between modeled and measured performance for a dense linear layer. (a) Correspondence over many dense linear layer configurations. The close correspondence across different bottleneck types (colors) provides evidence of the max-affine model's predictive power; the Pearson correlation coefficient is 0.996. (b) Roofline visualization for the dense linear layer over many configurations. NoC-bottlenecked configurations trace the roofline's sloped portion and SynOp-bottlenecked configurations trace the roofline's horizontal portion. All other points lie under the roofline. Please note that throughout this work error bars are omitted in linear layer workloads as over 3 trials, the measured mean time per step varied by at most 2%, and the microbenchmark system characteristic measurements underlying the max-affine model varied by at most 0.5%. †Workloads characterized on an Oheo Gulch system with a N3C2-revision Loihi 2 chip running an unreleased patch of NxCore 2.5.10 and NxKernel 0.4.0.