LVBA: LiDAR-Visual Bundle Adjustment for RGB Point Cloud Mapping

TL;DR

This work introduces a novel global LiDAR-Visual bundle adjustment (BA) named LVBA to improve the quality of RGB point cloud mapping beyond existing baselines and implements a novel LiDAR-assisted global visibility algorithm in LVBA.

Abstract

Point cloud maps with accurate color are crucial in robotics and mapping applications. Existing approaches for producing RGB-colorized maps are primarily based on real-time localization using filter-based estimation or sliding window optimization, which may lack accuracy and global consistency. In this work, we introduce a novel global LiDAR-Visual bundle adjustment (BA) named LVBA to improve the quality of RGB point cloud mapping beyond existing baselines. LVBA first optimizes LiDAR poses via a global LiDAR BA, followed by a photometric visual BA incorporating planar features from the LiDAR point cloud for camera pose optimization. Additionally, to address the challenge of map point occlusions in constructing optimization problems, we implement a novel LiDAR-assisted global visibility algorithm in LVBA. To evaluate the effectiveness of LVBA, we conducted extensive experiments by comparing its mapping quality against existing state-of-the-art baselines (i.e., R$^3$LIVE and FAST-LIVO). Our results prove that LVBA can proficiently reconstruct high-fidelity, accurate RGB point cloud maps, outperforming its counterparts.

Figures (8)

• Figure 1: An RGB point cloud map optimized using our method. The depicted data was captured at the Chong Yuet Ming Physics Building at The University of Hong Kong. Our method effectively optimizes both LiDAR and camera poses, achieving high levels of accuracy and consistency in the mapping process.
• Figure 2: The overview of our system. Our system consists of a LiDAR BA and a visual BA.
• Figure 3: LiDAR-assisted Global Visibility Map. A voxel map that stores the viability information of each voxel is constructed with LiDAR scans (\ref{['sec:global_visibility_map']}). After that, a global scene point is selected from all scene points in each voxel (Sec. \ref{['sec: global scene point selection']}). The visibility voxel map, together with the selected global scene point is then used in the global visibility determination process (Sec. \ref{['sec: Global Visibility Determination']})
• Figure 4: Photometric Error Formulation: A scene point $\boldsymbol \pi=(\mathbf p, \mathbf n)$ is first projected to the reference image frame at $\mathbf T_r$, and a reference patch $\{\mathbf u_r^{(i)}\}$ is generated. Then, the reference patch is projected and wrapped onto the target image frame at $\mathbf T_t$ by a homography transformation $\mathbf H$ to generate a target patch $\{\mathbf u_t^{(i)}\}$. Finally, a photometric error is constructed with the $L2$-norm of the radiance error between two patches.
• Figure 5: A sample output from our mapping evaluation algorithm. (a) displays the original image captured by the camera. (b) shows the image as rendered by our algorithm, and (c) presents the depth map, derived from the point cloud, which was used in the rendering of image (b). The rendered image (b) is then compared to the original (a) for evaluation using metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM).