Leveraging Cycle-Consistent Anchor Points for Self-Supervised RGB-D Registration

TL;DR

The paper addresses the challenge of self-supervised RGB-D registration with unlabeled data by leveraging cycle-consistent anchor points that are easily identifiable across views to impose spatial coherence on correspondences. It introduces a pixel-level matching pipeline that incorporates an anchor-point distance embedding, a self-attention mechanism, and a RANSAC-free pose refinement that combines a GRU-based updater with transformation synchronization in $SE(3)$. Training uses a registration loss and cycle-consistency losses to regularize anchor points, while test-time operation relies on Kabsch without the GRU or synchronization. Empirically, the method achieves state-of-the-art performance among self-supervised RGB-D registration approaches on ScanNet and 3DMatch and can improve other registration pipelines when integrated, highlighting its practical value for robust scene understanding from RGB-D data.

Abstract

With the rise in consumer depth cameras, a wealth of unlabeled RGB-D data has become available. This prompts the question of how to utilize this data for geometric reasoning of scenes. While many RGB-D registration meth- ods rely on geometric and feature-based similarity, we take a different approach. We use cycle-consistent keypoints as salient points to enforce spatial coherence constraints during matching, improving correspondence accuracy. Additionally, we introduce a novel pose block that combines a GRU recurrent unit with transformation synchronization, blending historical and multi-view data. Our approach surpasses previous self- supervised registration methods on ScanNet and 3DMatch, even outperforming some older supervised methods. We also integrate our components into existing methods, showing their effectiveness.

Figures (5)

• Figure 1: We propose to initially learn cycle-consistent salient points from RGB-D clips to substantially improve registration accuracy by subsequently constraining the correspondence estimation process.
• Figure 2: Illustration of Spatial Coherence The uniquely colored stars are anchor points, the black dot a correspondence and arrows indicate distance. (a) shows a spatially coherent correspondence. In the transformed view, the distances between the correspondence and the anchor point are roughly identical. (b) shows a spatially incoherent correspondence, with the red arrows indicating distances that violate spatial coherence.
• Figure 3: Overview of our Method Features are extracted from an RGB-D clip a via ResNet backbone. They are then used to learn a set of anchor points that are cycle-consistent across frames from the clip. These cycle-consistent anchor points are input into the pixel matching module. The pixel matching module and pose refinement module iterate feeding into each other. This is termed the inner iteration. The outer iteration happens periodically to update the anchor point locations. ADPE stands for Anchor Point Distance Embedding.
• Figure 4: Correspondence Ablation Ablation Study for Inlier % under various thresholds in 2D and 3D. While swapping or removing all four modules brings a decrease in inlier count, GC has the most significant effect followed by SC.
• Figure 5: Addition Analysis on Correspondence Error Adding SC+DE to SyncM has an even more significant impact on inlier % than even GC. All modules do improve the performance of SyncM.