RLCNet: An end-to-end deep learning framework for simultaneous online calibration of LiDAR, RADAR, and Camera
Hafeez Husain Cholakkal, Stefano Arrigoni, Francesco Braghin
TL;DR
RLCNet tackles the challenge of online, simultaneous extrinsic calibration for LiDAR, RADAR, and camera in dynamic driving environments. It introduces an end-to-end deep learning framework with five architectural modules and a differentiable optimization stage that enforces loop-closure consistency via a message-passing network, plus an online calibration framework using weighted moving averages and outlier rejection. The approach is trained through iterative refinement with decreasing miscalibration ranges and multiple loss terms, and validated on real-world VoD data and other baselines, achieving sub-degree rotations and centimeter-level translations. The combination of soft-feature sharing, loop-closure optimization, and online temporal filtering enables robust, real-time calibration suitable for deployment in autonomous systems.
Abstract
Accurate extrinsic calibration of LiDAR, RADAR, and camera sensors is essential for reliable perception in autonomous vehicles. Still, it remains challenging due to factors such as mechanical vibrations and cumulative sensor drift in dynamic environments. This paper presents RLCNet, a novel end-to-end trainable deep learning framework for the simultaneous online calibration of these multimodal sensors. Validated on real-world datasets, RLCNet is designed for practical deployment and demonstrates robust performance under diverse conditions. To support real-time operation, an online calibration framework is introduced that incorporates a weighted moving average and outlier rejection, enabling dynamic adjustment of calibration parameters with reduced prediction noise and improved resilience to drift. An ablation study highlights the significance of architectural choices, while comparisons with existing methods demonstrate the superior accuracy and robustness of the proposed approach.