PeLiCal: Targetless Extrinsic Calibration via Penetrating Lines for RGB-D Cameras with Limited Co-visibility

TL;DR

PeLiCal addresses extrinsic calibration for RGB-D camera rigs with limited co-visibility by using penetrating line features from the environment in a targetless, real-time framework. It formulates a merged quadratic system that couples full 3D and PnL constraints and parameterizes rotation with the CGR representation, refined via Levenberg–Marquardt while ensuring consistency with the $SO(3)$ manifold. Translation existence is validated through Plücker-coordinate-based constraints, with a convergence voting scheme robustly rejecting outliers. The approach demonstrates strong accuracy across varying field-of-view overlaps and baselines, outperforming or matching pattern-based methods while avoiding calibration targets and external devices; the implementation is open-source at https://github.com/joomeok/PeLiCal.git.

Abstract

RGB-D cameras are crucial in robotic perception, given their ability to produce images augmented with depth data. However, their limited FOV often requires multiple cameras to cover a broader area. In multi-camera RGB-D setups, the goal is typically to reduce camera overlap, optimizing spatial coverage with as few cameras as possible. The extrinsic calibration of these systems introduces additional complexities. Existing methods for extrinsic calibration either necessitate specific tools or highly depend on the accuracy of camera motion estimation. To address these issues, we present PeLiCal, a novel line-based calibration approach for RGB-D camera systems exhibiting limited overlap. Our method leverages long line features from surroundings, and filters out outliers with a novel convergence voting algorithm, achieving targetless, real-time, and outlier-robust performance compared to existing methods. We open source our implementation on https://github.com/joomeok/PeLiCal.git.

Figures (8)

• Figure 1: According to the inlier ratio of RANSAC-based fitting from the depth images, each line pair is classified into a full 3D case or PnL case, and different constraints are employed (top). Two checkerboard planes from each camera are merged by extrinsic parameters estimated from PeLiCal (bottom). Our method uses long lines matched from both images, and on the condition that the cameras are set up to observe the same feature, it can reliably determine the pose in cases with sufficient overlap (left) as well as in cases without any overlap (right).
• Figure 2: Line features (depicted in red) can be matched between two cameras even when they lack overlapping FOV areas. In contrast to point features, long lines can identify corresponding pairs under these conditions, allowing for determining geometric constraints.
• Figure 3: Pipeline of the proposed calibration algorithm. The procedure iteratively continues until the optimized solution reaches convergence and its associated cost drops below a predefined threshold.
• Figure 4: The results of the convergence voting process with candidate lines projected onto the $xy$-plane: (a) The algorithm successfully computes a convergence point, distinguishing inliers from outliers. (b) The candidate lines for translation do not converge due to an inaccurate rotation matrix or errors in the measurement.
• Figure 5: Equipment for accurate variation of rotation and translation between cameras. In the calibration process of our algorithm, the edge surface of the desk was used as a penetrating line.