NeuroVE: Brain-inspired Linear-Angular Velocity Estimation with Spiking Neural Networks

TL;DR

This letter proposes a brain-inspired framework for linear-angular velocity estimation, dubbed NeuroVE, and designs an Astrocyte Leaky Integrate-and-Fire neuron model to encode continuous values and develops an Astrocyte Spiking Long Short-term Memory structure, which significantly improves the time-series forecasting capabilities.

Abstract

Vision-based ego-velocity estimation is a fundamental problem in robot state estimation. However, the constraints of frame-based cameras, including motion blur and insufficient frame rates in dynamic settings, readily lead to the failure of conventional velocity estimation techniques. Mammals exhibit a remarkable ability to accurately estimate their ego-velocity during aggressive movement. Hence, integrating this capability into robots shows great promise for addressing these challenges. In this paper, we propose a brain-inspired framework for linear-angular velocity estimation, dubbed NeuroVE. The NeuroVE framework employs an event camera to capture the motion information and implements spiking neural networks (SNNs) to simulate the brain's spatial cells' function for velocity estimation. We formulate the velocity estimation as a time-series forecasting problem. To this end, we design an Astrocyte Leaky Integrate-and-Fire (ALIF) neuron model to encode continuous values. Additionally, we have developed an Astrocyte Spiking Long Short-term Memory (ASLSTM) structure, which significantly improves the time-series forecasting capabilities, enabling an accurate estimate of ego-velocity. Results from both simulation and real-world experiments indicate that NeuroVE has achieved an approximate 60% increase in accuracy compared to other SNN-based approaches.

Figures (7)

• Figure 1: The linear and angular speed estimation pipeline mirrors the processes of the human brain and robotic systems. In the vision circuits, the sensor transforms external light into spike signals. In the motion circuits, these signals are first encoded with temporal information. Subsequently, linear speed (LS) and angular speed (AS) cells translate these spikes into velocity signals.
• Figure 2: NeuroVE framework. The event data is denoted as $(t, x, y, p)$ and partitioned into $n$ chronological bins. These events are processed by spike coding and transformed into time-series spikes $[T, B, C \times n, H, W]$, where $T$ represents the time step, $B$ represents the batch size, $C$ represents the number of channels, $n$ represents the number of pieces into which the events are chronologically divided, $H$ and $W$ is the height and width, respectively. Finally, these time-series spikes are processed by the spiking feature extractor and the velocity estimator to infer the linear and angular velocities directly.
• Figure 3: The illustration describes the diffusion mechanism of membrane potentials in the two-dimensional computational graph of SNNs. Here, the blue block represents the LIF neurons, and the pink block represents the astrocyte-inspired diffusion mechanism.
• Figure 4: Elaborate on the mechanism by which astrocytes diffuse membrane potentials and illustrate the integration process with the LIF neurons within the ASLSTM.
• Figure 5: The graph of the results of numeric regression and time-series forecasting for $y = sin(x)$.