MULi-Ev: Maintaining Unperturbed LiDAR-Event Calibration

TL;DR

The findings reveal the potential of MULi-Ev to bolster the safety, reliability, and overall performance of perception systems in autonomous driving, marking a significant step forward in their real-world deployment and effectiveness.

Abstract

Despite the increasing interest in enhancing perception systems for autonomous vehicles, the online calibration between event cameras and LiDAR - two sensors pivotal in capturing comprehensive environmental information - remains unexplored. We introduce MULi-Ev, the first online, deep learning-based framework tailored for the extrinsic calibration of event cameras with LiDAR. This advancement is instrumental for the seamless integration of LiDAR and event cameras, enabling dynamic, real-time calibration adjustments that are essential for maintaining optimal sensor alignment amidst varying operational conditions. Rigorously evaluated against the real-world scenarios presented in the DSEC dataset, MULi-Ev not only achieves substantial improvements in calibration accuracy but also sets a new standard for integrating LiDAR with event cameras in mobile platforms. Our findings reveal the potential of MULi-Ev to bolster the safety, reliability, and overall performance of event-based perception systems in autonomous driving, marking a significant step forward in their real-world deployment and effectiveness.

Figures (4)

• Figure 1: Overview of the MULi-Ev calibration workflow. This process integrates LiDAR point clouds and event camera data into the MULi-Ev network to compute accurate extrinsic calibration parameters (the rigid transformation in $SO(3)$ between the two sensors' reference frames, here represented by T). These parameters enable real-time, precise sensor alignment, facilitating enhanced perception for autonomous vehicles in dynamic scenarios.
• Figure 2: Overall architecture of MULi-Ev. The initial decalibrated extrinsic parameters T (three for rotation, and three for translation) are used to project the LiDAR point cloud into the event camera frame. Both input are then fed to a MobileViTv2 mehta2022separable backbone for feature extraction. The features are passed to a regression head, which regresses separately translation and rotation parameters. Together, they compose the output T, which the loss $\mathcal{L}$ compares to the known ground truth.
• Figure 3: Box plots of translation and rotation errors on the test set of DSEC gehrig2021dsec.
• Figure 4: Qualitative results on DSEC gehrig2021dsec, showing three examples of recalibration in diverse environments. Images show the LiDAR pointclouds projected on the event frame. The top lane presents random decalibrations applied to the setup, while the bottom lane presents the correction proposed by MULi-Ev.