THE-SEAN: A Heart Rate Variation-Inspired Temporally High-Order Event-Based Visual Odometry with Self-Supervised Spiking Event Accumulation Networks

TL;DR

THE-SEAN addresses the inefficiency of fixed-trigger, zero-order event-based VO by introducing a temporally high-order, self-supervised triggering framework. A bio-inspired spiking neural network (SEAN) autonomously learns when to map or track by regressing Q-values using rewards derived from the estimator itself, notably the Fisher information trace and valid block matches. This results in dynamic trigger timing, improved estimation accuracy and smoothness, and substantially reduced triggering costs across multiple datasets. The approach demonstrates strong potential for low-power, real-time operation in fast-moving or complex environments, with clear avenues toward neuromorphic hardware deployment and broader sensor fusion integration.

Abstract

Event-based visual odometry has recently gained attention for its high accuracy and real-time performance in fast-motion systems. Unlike traditional synchronous estimators that rely on constant-frequency (zero-order) triggers, event-based visual odometry can actively accumulate information to generate temporally high-order estimation triggers. However, existing methods primarily focus on adaptive event representation after estimation triggers, neglecting the decision-making process for efficient temporal triggering itself. This oversight leads to the computational redundancy and noise accumulation. In this paper, we introduce a temporally high-order event-based visual odometry with spiking event accumulation networks (THE-SEAN). To the best of our knowledge, it is the first event-based visual odometry capable of dynamically adjusting its estimation trigger decision in response to motion and environmental changes. Inspired by biological systems that regulate hormone secretion to modulate heart rate, a self-supervised spiking neural network is designed to generate estimation triggers. This spiking network extracts temporal features to produce triggers, with rewards based on block matching points and Fisher information matrix (FIM) trace acquired from the estimator itself. Finally, THE-SEAN is evaluated across several open datasets, thereby demonstrating average improvements of 13\% in estimation accuracy, 9\% in smoothness, and 38\% in triggering efficiency compared to the state-of-the-art methods.

Figures (4)

• Figure 1: Comparison of the proposed temporally high-order event-based visual odometry system with the traditional zero-order event-based estimator. Traditional estimators typically rely on constant-frequency triggers with adaptive accumulation, resulting in computational redundancy and noise accumulation. In contrast, our proposed temporally high-order system dynamically determines the optimal trigger moments, which is inspired by heart rate variation mechanism of human, thereby enhancing both computational efficiency and estimation accuracy.
• Figure 2: Problem formulation of temporally high-order event-based estimator. The asynchronous estimator must determine when to trigger the mapping and tracking process in order to minimize both estimation error and power consumption.
• Figure 3: System overview of the bio-inspired temporally high-order estimation framework, THE-SEAN. The left green section illustrates the biological mechanism for asynchronous state estimation, where sensory spike signals pass through neurons, triggering hormone secretion that regulates heart rate. Dopamine, as a reward signal, adjusts hormone levels in a feedback loop. The right blue section shows the proposed THE-SEAN, which emulates this process. Event spikes are processed by spiking neural networks, and Q-values are regressed to regulate the trigger rate. The rewards acquired from the estimator itself, including Fisher information matrix (FIM) trace and valid block matching points, supervise the network weights in a closed-loop system. The corresponding processes are color-coded by A, B, C, D for clarity.
• Figure 4: Trajectory comparison and MTR analysis. (a) and (b) illustrate part of the representative estimated trajectories by THE-SEAN and baselines on MVSEC. (c) shows the MTR variation of THE-SEAN corresponding to theagent velocity in sequence indoor1.