A Target-based Multi-LiDAR Multi-Camera Extrinsic Calibration System

TL;DR

The paper tackles extrinsic calibration for a heterogeneous set of sensors comprising multiple LiDARs and cameras in autonomous systems. It introduces a target-based framework using a custom anchor board detectable by both modalities and a unified nonlinear optimization that aligns all sensors to a common reference frame by minimizing reprojection residuals of the target features. The approach integrates camera 2D detections (ChArUco/ArUco) and LiDAR depth discontinuities across camera-camera, LiDAR-camera, and LiDAR-LiDAR relationships, formulated within a single optimization problem. Experimental evaluation in a warehouse setting with three cameras and two LiDARs demonstrates accurate, repeatable inter-sensor transformations and a practical pipeline for robust cross-modal calibration.

Abstract

Extrinsic Calibration represents the cornerstone of autonomous driving. Its accuracy plays a crucial role in the perception pipeline, as any errors can have implications for the safety of the vehicle. Modern sensor systems collect different types of data from the environment, making it harder to align the data. To this end, we propose a target-based extrinsic calibration system tailored for a multi-LiDAR and multi-camera sensor suite. This system enables cross-calibration between LiDARs and cameras with limited prior knowledge using a custom ChArUco board and a tailored nonlinear optimization method. We test the system with real-world data gathered in a warehouse. Results demonstrated the effectiveness of the proposed method, highlighting the feasibility of a unique pipeline tailored for various types of sensors.

Figures (6)

• Figure 1: Projection of the board centers found by a LiDAR (red cross) and a camera (green dot). Blue IDs are the ChArUCo marker detections from the camera.
• Figure 1: Compact Calibration Results to Camera 0
• Figure 2: Design of the target board.
• Figure 3: Calibration scheme of a generic Camera-LiDAR system with related frames.
• Figure 4: Projection of every center detected by each sensor in the image collected by the reference camera. The red square is related to the first camera that detects the target. The blue and green squares are centers detected by the two LiDARs.