A Target-Based Extrinsic Calibration Framework for Non-Overlapping Camera-Lidar Systems Using a Motion Capture System

TL;DR

This paper addresses the challenge of calibrating extrinsic transforms between non-overlapping lidar and camera sensors by introducing a target-based framework that uses a Motion Capture System (MCS) to track sensor and target poses. The method estimates $T_{RL}$ and $T_{RC}$ in a common map frame using MCS-derived measurements ($T_{MT}$, $T_{MR}$) and a target-alignment term $T_E$, with a target-agnostic, nearest-neighbor data association and a single Levenberg–Marquardt optimization to minimize the combined lidar and camera reprojection/point-to-point costs. It demonstrates through simulation and real experiments that the approach achieves higher accuracy and repeatability than state-of-the-art targetless methods, while removing dependence on overlapping FOVs and mitigating error propagation from chained calibrations. The work also shows robustness to different target designs (e.g., diamond, cylinder) and provides an extensible open-source implementation, enabling practical, high-precision sensor fusion for diverse robotic applications.

Abstract

We present a novel target-based lidar-camera extrinsic calibration methodology that can be used for non-overlapping field of view (FOV) sensors. Contrary to previous work, our methodology overcomes the non-overlapping FOV challenge using a motion capture system (MCS) instead of traditional simultaneous localization and mapping approaches. Due to the high relative precision of MCSs, our methodology can achieve both the high accuracy and repeatable calibrations common to traditional target-based methods, regardless of the amount of overlap in the sensors' field of view. Furthermore, we design a target-agnostic implementation that does not require uniquely identifiable features by using an iterative closest point approach, enabled by the MSC measurements. We show using simulation that we can accurately recover extrinsic calibrations for a range of perturbations to the true calibration that would be expected in real circumstances. We prove experimentally that our method out-performs state-of-the-art lidar-camera extrinsic calibration methods that can be used for non-overlapping FOV systems, while using a target-based approach that guarantees repeatably high accuracy. Lastly, we show in simulation that different target designs can be used, including easily constructed 3D targets such as a cylinder that are normally considered degenerate in most calibration formulations.

Figures (9)

• Figure 1: Illustration of the proposed calibration framework with Motion Capture System
• Figure 2: Calibration problem with all coordinate frames
• Figure 3: Calibration process overview
• Figure 4: Example correspondence estimation for diamond target: camera keypoints (left), lidar keypoints (right). Red: measured keypoints. Green: transformed ground truth keypoints
• Figure 5: Example lidar calibration (left) and camera calibration (right) results using simulation data with diamond target