Targetless Intrinsics and Extrinsic Calibration of Multiple LiDARs and Cameras with IMU using Continuous-Time Estimation

TL;DR

This work tackles the problem of targetless, joint spatiotemporal calibration of multiple LiDARs, cameras, and an IMU under sensor asynchrony and non-overlapping FOVs. It introduces a continuous-time bundle adjustment framework that jointly estimates $6$-DOF extrinsics and time offsets, camera intrinsics, and IMU parameters by leveraging SFM-derived camera co-visibility, voxel-map plane BA for LiDARs, and cross-modal LiDAR-camera intensity alignment, all without calibration boards. Key innovations include continuous-time trajectory representation with Lie-group-aware B-splines, LiDAR plane BA with adaptive voxel maps, and a cross-modal LiDAR-camera data association for final joint optimization. The method is validated on a real autonomous-driving dataset, achieving high-precision extrinsics and intrinsics across numerous sensors and demonstrating reduced calibration drift compared with conventional, per-sensor calibration pipelines, enabling accurate multisensor fusion in practical settings.

Abstract

Accurate spatiotemporal calibration is a prerequisite for multisensor fusion. However, sensors are typically asynchronous, and there is no overlap between the fields of view of cameras and LiDARs, posing challenges for intrinsic and extrinsic parameter calibration. To address this, we propose a calibration pipeline based on continuous-time and bundle adjustment (BA) capable of simultaneous intrinsic and extrinsic calibration (6 DOF transformation and time offset). We do not require overlapping fields of view or any calibration board. Firstly, we establish data associations between cameras using Structure from Motion (SFM) and perform self-calibration of camera intrinsics. Then, we establish data associations between LiDARs through adaptive voxel map construction, optimizing for extrinsic calibration within the map. Finally, by matching features between the intensity projection of LiDAR maps and camera images, we conduct joint optimization for intrinsic and extrinsic parameters. This pipeline functions in texture-rich structured environments, allowing simultaneous calibration of any number of cameras and LiDARs without the need for intricate sensor synchronization triggers. Experimental results demonstrate our method's ability to fulfill co-visibility and motion constraints between sensors without accumulating errors.

Figures (11)

• Figure 1: Factor graphs. We achieve joint optimization of continuous-time trajectort, extrinsic and intrinsic parameters by combining visual measurements, LiDAR measurements, IMU measurements and LiDAR-visual measurements
• Figure 2: System Pipeline. Each block is explained in the corresponding subsection of \ref{['section:4']} with explicit reference back to this diagram.
• Figure 3: LiDAR BA calibration. a) Before calibration, initialization errors caused layering in the point cloud, making it challenging for multiple LiDARs to collectively establish a voxel map. b) After single LiDAR BA calibration, the consistency of LiDAR point cloud is enhanced, but there is still a little stratification. c) Multi-LiDARs co-construction of the voxel map. d) After calibration, the layering in the point cloud disappears, aligning it with the IMU trajectory, resulting in improved map consistency.
• Figure 4: Pose and extrinsic parameters refined with LiDAR point cloud in purple and visual point cloud map in white.
• Figure 5: a) Original image. b) Projection and rendering of the LiDAR map onto the imaging plane based on camera pose. c) Matching results between the camera, original image, and intensity map.