Mamba-Spike: Enhancing the Mamba Architecture with a Spiking Front-End for Efficient Temporal Data Processing

TL;DR

This work tackles the challenge of efficient temporal data processing for event-based and time-series inputs by introducing Mamba-Spike, a hybrid architecture that couples a spiking front-end with the Mamba backbone. The front-end encodes temporal data into sparse spike representations using biologically inspired neuron models, while the backbone leverages selective state spaces and linear-time sequence modeling to capture long-range temporal dependencies. Extensive experiments on neuromorphic datasets (DVS Gesture, TIDIGITS) and standard benchmarks (Sequential MNIST, CIFAR10-DVS) show state-of-the-art accuracy, reduced latency, and improved energy efficiency, with ablations confirming the benefits of the spiking front-end and the choice of neuron models. The results suggest Mamba-Spike as a practical, robust solution for real-world temporal tasks, with code released for reproducibility.

Abstract

The field of neuromorphic computing has gained significant attention in recent years, aiming to bridge the gap between the efficiency of biological neural networks and the performance of artificial intelligence systems. This paper introduces Mamba-Spike, a novel neuromorphic architecture that integrates a spiking front-end with the Mamba backbone to achieve efficient and robust temporal data processing. The proposed approach leverages the event-driven nature of spiking neural networks (SNNs) to capture and process asynchronous, time-varying inputs, while harnessing the power of the Mamba backbone's selective state spaces and linear-time sequence modeling capabilities to model complex temporal dependencies effectively. The spiking front-end of Mamba-Spike employs biologically inspired neuron models, along with adaptive threshold and synaptic dynamics. These components enable efficient spatiotemporal feature extraction and encoding of the input data. The Mamba backbone, on the other hand, utilizes a hierarchical structure with gated recurrent units and attention mechanisms to capture long-term dependencies and selectively process relevant information. To evaluate the efficacy of the proposed architecture, a comprehensive empirical study is conducted on both neuromorphic datasets, including DVS Gesture and TIDIGITS, and standard datasets, such as Sequential MNIST and CIFAR10-DVS. The results demonstrate that Mamba-Spike consistently outperforms state-of-the-art baselines, achieving higher accuracy, lower latency, and improved energy efficiency. Moreover, the model exhibits robustness to various input perturbations and noise levels, highlighting its potential for real-world applications. The code will be available at https://github.com/ECNU-Cross-Innovation-Lab/Mamba-Spike.

Figures (5)

• Figure 1: The spiking front-end (left) processes raw temporal data and encodes it into sparse spike representations using biologically-inspired neuron models and synaptic dynamics. The Mamba backbone (right) leverages selective state spaces and linear-time sequence modelling to efficiently capture complex temporal dependencies. The integration of these two components enables efficient and robust processing of asynchronous, time-varying inputs, bridging the gap between the efficiency of spiking neural networks and the performance of conventional deep learning models.
• Figure 2: Overview of the Mamba-Spike architecture. The spiking front-end processes raw temporal data and encodes them into sparse spike representations, which are then fed into the Mamba backbone for efficient sequence modeling.
• Figure 3: Detailed view of the spiking front-end module. Raw temporal data are encoded using event-based encoding schemes and processed by spiking neuron models. Spatial-temporal feature extraction is performed to generate a compact spike representation for the Mamba backbone.
• Figure 4: The Sequential MNIST dataset is converted into time surfaces.
• Figure 5: Effect of spiking neuron models and time constants on the accuracy of the Mamba-Spike architecture for the TIDIGITS dataset.