SGBA: Semantic Gaussian Mixture Model-Based LiDAR Bundle Adjustment

TL;DR

This work tackles drift and generalization challenges in LiDAR bundle adjustment by introducing SGBA, which models the environment as a semantic Gaussian Mixture Model to fuse geometric and semantic cues without predefined landmarks. It integrates an adaptive semantic label selection framework based on the condition number of the optimization and a probabilistic feature association scheme to robustly handle measurement and initialization uncertainties. The approach yields improved pose accuracy and robustness across diverse datasets and LiDAR types, with controlled computational complexity through selective semantic layers. The method is open-source, enabling community validation and extension for broader autonomous-system applications.

Abstract

LiDAR bundle adjustment (BA) is an effective approach to reduce the drifts in pose estimation from the front-end. Existing works on LiDAR BA usually rely on predefined geometric features for landmark representation. This reliance restricts generalizability, as the system will inevitably deteriorate in environments where these specific features are absent. To address this issue, we propose SGBA, a LiDAR BA scheme that models the environment as a semantic Gaussian mixture model (GMM) without predefined feature types. This approach encodes both geometric and semantic information, offering a comprehensive and general representation adaptable to various environments. Additionally, to limit computational complexity while ensuring generalizability, we propose an adaptive semantic selection framework that selects the most informative semantic clusters for optimization by evaluating the condition number of the cost function. Lastly, we introduce a probabilistic feature association scheme that considers the entire probability density of assignments, which can manage uncertainties in measurement and initial pose estimation. We have conducted various experiments and the results demonstrate that SGBA can achieve accurate and robust pose refinement even in challenging scenarios with low-quality initial pose estimation and limited geometric features. We plan to open-source the work for the benefit of the community https://github.com/Ji1Xinyu/SGBA.

Figures (6)

• Figure 1: Comparison of different landmark representation methods and their BA performance.
• Figure 2: Workflow of SGBA. The system first takes the aggregated cloud as input and layers the cloud based on semantic labels. Afterward, initial Gaussians are fitted separately on each semantic layer. An initial optimization problem is constructed and evaluated for degeneracy. If degeneracy is detected, additional semantic layers are included. Finally, the optimal robot poses are estimated by iteratively solving the MLE problem \ref{['eq_mle_ori']}.
• Figure 3: Comparisons of feature association methods (different colors denote different semantic labels). (a). $\Bar{\boldsymbol{T}}_k\boldsymbol{z}_{ki}$ is directly assigned to the nearest but wrong landmark $\boldsymbol{l}_1$. (b). $\Bar{\boldsymbol{T}}_k\boldsymbol{z}_{ki}$ is soft-assigned to $\boldsymbol{l}_0$ and $\boldsymbol{l}_3$ with different probability.
• Figure 4: Scene dominated by degenerative planar features causing geometric-only LiDAR BA to fail. (a). Original point cloud. (b). Planes extracted by plane-based BA methods. No planes can provide horizontal constraints.
• Figure 5: Left: SLICT, Middle: BALM, Right: SGBA. Map comparison on kth_05 sequence using MCD semantic coloring. SGBA shows better generalization for the challenging outdoor curved scene.