Event-based Visual Inertial Velometer

TL;DR

The paper tackles robust state estimation for aggressive ego-motion by replacing pose estimation with instantaneous linear velocity estimation using a map-free, event-based visual–inertial velometer. It fuses stereo event camera data and an IMU in a continuous-time framework, modeling velocity with a cubic B-spline and enforcing both event-derived normal-flow constraints and IMU pre-integration corrections. Empirical results on synthetic and real spinning datasets show metric-scale velocity with low latency and improved dead-reckoning over frame-based VO baselines, highlighting robustness to motion blur and high-speed dynamics. By aligning estimation with the differential nature of event data, the approach reduces data-association dependence and offers a practical path toward reliable high-speed navigation with neuromorphic vision.

Abstract

Neuromorphic event-based cameras are bio-inspired visual sensors with asynchronous pixels and extremely high temporal resolution. Such favorable properties make them an excellent choice for solving state estimation tasks under aggressive ego motion. However, failures of camera pose tracking are frequently witnessed in state-of-the-art event-based visual odometry systems when the local map cannot be updated in time. One of the biggest roadblocks for this specific field is the absence of efficient and robust methods for data association without imposing any assumption on the environment. This problem seems, however, unlikely to be addressed as in standard vision due to the motion-dependent observability of event data. Therefore, we propose a mapping-free design for event-based visual-inertial state estimation in this paper. Instead of estimating the position of the event camera, we find that recovering the instantaneous linear velocity is more consistent with the differential working principle of event cameras. The proposed event-based visual-inertial velometer leverages a continuous-time formulation that incrementally fuses the heterogeneous measurements from a stereo event camera and an inertial measurement unit. Experiments on the synthetic dataset demonstrate that the proposed method can recover instantaneous linear velocity in metric scale with low latency.

Figures (8)

• Figure 1: The proposed system takes as input the event data from a stereo event-based camera and inertial measurements from an IMU. (a) Aggressive maneuvers of a drone through a narrow corridor. (b) Event-based normal flow estimates. (c) Corresponding depth estimates. (d) Illustration of the normalized result of instantaneous linear velocity estimation.
• Figure 2: Geometry and kinematics involved in the problem of motion flow. An ideal perspective camera model is used to illustrate that the 3D point $\mathbf{P}$ is projected on the image plane. The components of both linear velocity and angular velocity are marked along their corresponding axis. The resulting motion flow is denoted by the red vector $(\dot{x},\dot{y})$.
• Figure 3: General principles of normal flow estimation from raw event streams. (a) A time surface $\mathcal{T}(x,y)$ is spanned in the spatio-temporal domain as the edge (black) traverses. The temporal flow $\nabla_{\mathbf{x}}\mathcal{T}$ can be estimated by fitting a local plane. A color scale for the temporal values is provided aside. The darker the color, the closer the time is to the current moment. (b) The normal flow $\dot{\mathbf{x}}_n$, namely the component of the motion flow $\dot{\mathbf{x}}$ along the direction of brightness gradient $\mathbf{n}$, is parallel to the temporal flow $\nabla_{\mathbf{x}}\mathcal{T}$.
• Figure 4: Flowchart of the proposed event-based visual-inertial velometer system. The system takes as input the events from a stereo event camera and an IMU's inertial measurements, and reports the estimated linear velocity.
• Figure 5: Simulated scenes for synthetic data generation.