SGOR: Outlier Removal by Leveraging Semantic and Geometric Information for Robust Point Cloud Registration

TL;DR

SGOR addresses robust point cloud registration under semantic and geometric imperfections by integrating secondary ground segmentation, regional voting-based loose semantic consistency, and a semantic-geometric consistency framework for outlier removal. A two-stage hypothesis verification guided by ground priors further improves robustness in weak-geometry scenes, followed by SVD-based refinement. Empirical results on KITTI show state-of-the-art registration recall (RR=98.92%) with low rotation and translation errors, and strong outlier-filtering performance, while 3DMatch tests demonstrate resilience in indoor, partially overlapping scenes. The approach significantly narrows the gap between semantic priors and geometric cues, offering practical robustness for outdoor and indoor registration scenarios.

Abstract

In this paper, we introduce a new outlier removal method that fully leverages geometric and semantic information, to achieve robust registration. Current semantic-based registration methods only use semantics for point-to-point or instance semantic correspondence generation, which has two problems. First, these methods are highly dependent on the correctness of semantics. They perform poorly in scenarios with incorrect semantics and sparse semantics. Second, the use of semantics is limited only to the correspondence generation, resulting in bad performance in the weak geometry scene. To solve these problems, on the one hand, we propose secondary ground segmentation and loose semantic consistency based on regional voting. It improves the robustness to semantic correctness by reducing the dependence on single-point semantics. On the other hand, we propose semantic-geometric consistency for outlier removal, which makes full use of semantic information and significantly improves the quality of correspondences. In addition, a two-stage hypothesis verification is proposed, which solves the problem of incorrect transformation selection in the weak geometry scene. In the outdoor dataset, our method demonstrates superior performance, boosting a 22.5 percentage points improvement in registration recall and achieving better robustness under various conditions. Our code is available.

Figures (6)

• Figure 1: Our method can also achieve robust registration even in the case of simple geometry and semantic inaccuracy. The green line is the correct correspondence and the red line is the wrong correspondence.
• Figure 2: Pipeline of our proposed method. First, we input the point cloud and its semantics to construct a semantic-geometric space $\mathbb{U}$. Next, the secondary ground segmentation separates the point cloud into ground $\mathbf{U}_{\mathbf{P}g}$ and non-ground points $\mathbf{U}_{\mathbf{P}n}$ . Subsequently, we estimate the overlap region $\mathbf{S}_{\mathbf{P}_g}$ within the semantic space, extract point features for feature matching, and establish the correspondences $\mathcal{G}$. After that, we obtain local correspondences $\mathcal{G}^k$ by sampling and grouping. Outliers within each local correspondence are filtered based on semantic-geometric consistency, and local transformations $\mathbf{T}_l$ are estimated. Finally, the optimal transformation $\tilde{ \mathbf{R}}$ is selected through the two-stage hypothesis verification.
• Figure 3: Ground segmentation and verification with ground prior.
• Figure 4: Correspondences result and registration result on KITTI dataset.
• Figure 5: Qualitative results under indistinct geometric features and few semantic categories. Despite the state-of-the-art method chen2022sc2 and the semantic registration method qiao2023pyramid fail, our approach still achieves robust registration.