SuperEIO: Self-Supervised Event Feature Learning for Event Inertial Odometry

TL;DR

This paper proposes SuperEIO, a novel framework that leverages the learning-based event-only detection and IMU measurements to achieve event-inertial odometry and utilizes TensorRT to accelerate the inference speed of deep networks, which ensures low-latency processing and robust real-time operation on resource-limited platforms.

Abstract

Event cameras asynchronously output low-latency event streams, promising for state estimation in high-speed motion and challenging lighting conditions. As opposed to frame-based cameras, the motion-dependent nature of event cameras presents persistent challenges in achieving robust event feature detection and matching. In recent years, learning-based approaches have demonstrated superior robustness over traditional handcrafted methods in feature detection and matching, particularly under aggressive motion and HDR scenarios. In this paper, we propose SuperEIO, a novel framework that leverages the learning-based event-only detection and IMU measurements to achieve event-inertial odometry. Our event-only feature detection employs a convolutional neural network under continuous event streams. Moreover, our system adopts the graph neural network to achieve event descriptor matching for loop closure. The proposed system utilizes TensorRT to accelerate the inference speed of deep networks, which ensures low-latency processing and robust real-time operation on resource-limited platforms. Besides, we evaluate our method extensively on multiple public datasets, demonstrating its superior accuracy and robustness compared to other state-of-the-art event-based methods. We have also open-sourced our pipeline to facilitate research in the field: https://github.com/arclab-hku/SuperEIO.

Figures (8)

• Figure 1: An overview of the proposed SuperEIO system. We develop a self-supervised event detector that extracts feature points and descriptors from asynchronous events. To enable loop closure, our proposed event descriptor matcher establishes event correspondences. The entire system tightly integrates self-supervised event feature learning with IMU measurements and is optimized with TensorRT, achieving robust and real-time estimation.
• Figure 2: The architecture of the proposed event detector.
• Figure 3: Generation of simulated events from a single image. The left side illustrates the specific process: (1) selecting a random crop region, (2) translating, (3) rotating, (4) zooming the region, and (5) generating simulated events from the resized image pair $(I_1, I_2)$ using ESIM. Additional generated samples are presented on the right side.
• Figure 4: The architecture of our event descriptor matcher.
• Figure 5: Comparison of event-based feature detection methods on stereo HKU (above) and DAVIS240c datasets (below).