AsynEIO: Asynchronous Monocular Event-Inertial Odometry Using Gaussian Process Regression

TL;DR

AsynEIO addresses the challenge of fusing asynchronous event streams with IMU data for monocular odometry by casting the motion trajectory as a continuous-time Gaussian Process on $SE(3)$. It introduces multiple inertial fusion schemes, notably GP Inertial Factor (GPIF) and GP Preintegration (GPP), along with an Extended Preintegration variant to enable flexible, asynchronous back-end optimization in a sliding-window graph. An event-driven front-end tracks feature trajectories directly from raw events, while a reprojection-based reprojection mechanism and inverse-depth initialization support robust data association. Experimental results on public and own-collected data demonstrate improved accuracy and robustness in high-speed and low-illumination conditions, with practical guidance on choosing inertial schemes and a discussion of limitations and future work.

Abstract

Event cameras, when combined with inertial sensors, show significant potential for motion estimation in challenging scenarios, such as high-speed maneuvers and low-light environments. There are many methods for producing such estimations, but most boil down to a synchronous discrete-time fusion problem. However, the asynchronous nature of event cameras and their unique fusion mechanism with inertial sensors remain underexplored. In this paper, we introduce a monocular event-inertial odometry method called AsynEIO, designed to fuse asynchronous event and inertial data within a unified Gaussian Process (GP) regression framework. Our approach incorporates an event-driven frontend that tracks feature trajectories directly from raw event streams at a high temporal resolution. These tracked feature trajectories, along with various inertial factors, are integrated into the same GP regression framework to enable asynchronous fusion. With deriving analytical residual Jacobians and noise models, our method constructs a factor graph that is iteratively optimized and pruned using a sliding-window optimizer. Comparative assessments highlight the performance of different inertial fusion strategies, suggesting optimal choices for varying conditions. Experimental results on both public datasets and our own event-inertial sequences indicate that AsynEIO outperforms existing methods, especially in high-speed and low-illumination scenarios.

Figures (20)

• Figure 1: Illustration of two asynchronous fusion schemes (GPIF and GPP). The GPIF conducts a mass of factors for inertial measurements and deforms the continuous-time trajectory in conjunction with GP-based motion prior. The GPP first reduces a series of latent states and then integrates them efficiently with the linear operator at arbitrary timestamps.
• Figure 2: System framework of AsynEIO.
• Figure 3: Event-driven feature trajectory generation and management. The registration table (left) marks the active pixels patches, which limits the detection and tracking regions and enables an event-driven front-end (right). All potential event features (labeled as Discarded Fts) within the active Tracker Patch will be abandoned.
• Figure 4: Illustration of event streams and feature trajectories. When the event camera observes some landmarks and moves in the scenario, the event stream triggered by the same landmark will be managed as a feature trajectory.
• Figure 5: Inverse depth parametrization.