Spark: Modular Spiking Neural Networks

TL;DR

Spark addresses the inefficiency and learning challenges of traditional neural networks by introducing a modular, GPU-based framework for spiking neural networks that supports unbatched, iterative learning. It decomposes models into reusable neuronal, interface, and controller components and provides a blueprint-instantiation approach and a GUI to improve reproducibility and exploration. The authors demonstrate fidelity against Brian2 for common neuron models and show substantial interactive performance gains, while solving a sparse-reward Cartpole problem with simple, three-factor plasticity and no surrogate gradients. This work suggests that modular SNN tooling can accelerate research and enable more ML-like pipelines for SNNs, bringing iterative learning and continuous adaptation closer to brain-like dynamics.

Abstract

Nowadays, neural networks act as a synonym for artificial intelligence. Present neural network models, although remarkably powerful, are inefficient both in terms of data and energy. Several alternative forms of neural networks have been proposed to address some of these problems. Specifically, spiking neural networks are suitable for efficient hardware implementations. However, effective learning algorithms for spiking networks remain elusive, although it is suspected that effective plasticity mechanisms could alleviate the problem of data efficiency. Here, we present a new framework for spiking neural networks - Spark - built upon the idea of modular design, from simple components to entire models. The aim of this framework is to provide an efficient and streamlined pipeline for spiking neural networks. We showcase this framework by solving the sparse-reward cartpole problem with simple plasticity mechanisms. We hope that a framework compatible with traditional ML pipelines may accelerate research in the area, specifically for continuous and unbatched learning, akin to the one animals exhibit.

Figures (6)

• Figure 3: Spark's mascot.
• Figure 4: A glimpse into Spark's graphical interface. Source nodes represent inputs to the model, while sink nodes represent outputs. The graph editor allows for simple and quick model design. Currently not all features of Spark are supported through the editor but models can be imported/exported to allow for blueprint edition via code as necessary.
• Figure 5: Fidelity benchmark. Three soma simulations of increasing complexity, LIF (a,b), AdEX (c,d) and HH (e,f) are used to showcase the fidelity of Spark simulations. As a reference for comparison we use Brian2 and model implementations used in the official Brian2 documentation. Simulations are driven by an Ornstein–Uhlenbeck process-like current for one second. Left subplots (a, c, e) show the input current (top subplot), the membrane potential (middle subplot) and the output spikes (bottom subplot) for a single trial of 100 ms. Right subplots (b, d, f) show two common train spike statistics: the average ISI distance and the average SPIKE-distance for 100 simulations, one second each. For comparison, we use a shuffle version of the simulation spike trains.
• Figure 6: Performance benchmark of three different network architectures. (a,d,g) Network architectures used for the simulations. Each node represents a "pool of neurons" of 1024 units, 20% of which are inhibitory neurons. (b,e,h) Average relative speedup of 10 seconds simulations, 25 network instantiations, 5 runs per instantiation, for architectures in (a,d,g), respectively. Speedups are normalized with respect to a C++ single execution (higher is faster). Execution timesteps indicate how many timesteps each model computes before a new input signal is presented to the network and a new output signal is registered from the network. Dashed lines denote the best-case scenario, i.e., when the network is not interrupted, for Brian2 with the C++ and CUDA backends. (c,f,i) Average compile times for each instantiated network model, for architectures in (a,d,g), respectively. Compilation is considered as the time for object construction plus the C++/JIT compilation required to run the network; numpy is considered as the time to instantiate the network object.
• Figure 7: Cartpole model diagram. The model consists of a simple A vs B architecture; activity on A inhibits activity on B and vice versa. Agent input is passed through a topological Spiker which maps single values (e.g. an angle) into a distribution of spikes. Agent output consists of a traced "measurement" of the activity of each excitatory subpopulation.