Corr2Distrib: Making Ambiguous Correspondences an Ally to Predict Reliable 6D Pose Distributions

TL;DR

Corr2Distrib tackles the challenge of estimating a 6DoF camera pose distribution from a single RGB image by turning symmetry- and occlusion-induced ambiguities into multiple rotation hypotheses derived from 2D-3D correspondences. It learns a symmetry-aware representation (descriptors and local frames) per object surface point, discretizes the rotation space with an equi-volume grid, and refines hypotheses into a full $6D$ pose distribution using PnP-RANSAC and a joint descriptor/mask scoring. The approach achieves state-of-the-art performance on the BOP-Distrib benchmark for both pose distributions and single-pose estimates, illustrating the value of correspondences for distribution estimation and highlighting practical gains in robustness to symmetry-related ambiguities. The work introduces a scalable, correspondence-based framework that can inform downstream tasks such as robotic grasping and assembly by providing calibrated pose distributions, and it points to future directions in data, texture disentanglement, and attention-based refinement.

Abstract

We introduce Corr2Distrib, the first correspondence-based method which estimates a 6D camera pose distribution from an RGB image, explaining the observations. Indeed, symmetries and occlusions introduce visual ambiguities, leading to multiple valid poses. While a few recent methods tackle this problem, they do not rely on local correspondences which, according to the BOP Challenge, are currently the most effective way to estimate a single 6DoF pose solution. Using correspondences to estimate a pose distribution is not straightforward, since ambiguous correspondences induced by visual ambiguities drastically decrease the performance of PnP. With Corr2Distrib, we turn these ambiguities into an advantage to recover all valid poses. Corr2Distrib first learns a symmetry-aware representation for each 3D point on the object's surface, characterized by a descriptor and a local frame. This representation enables the generation of 3DoF rotation hypotheses from single 2D-3D correspondences. Next, we refine these hypotheses into a 6DoF pose distribution using PnP and pose scoring. Our experimental evaluations on complex non-synthetic scenes show that Corr2Distrib outperforms state-of-the-art solutions for both pose distribution estimation and single pose estimation from an RGB image, demonstrating the potential of correspondences-based approaches.

Figures (5)

• Figure 1: Pose distribution comparison of our method with SpyroPose haugaard2023spyropose and LiePose hsiao2024confronting on the T-LESS dataset hodan2017t. We illustrate three cases of the object 1 from top to bottom: no symmetry, six-fold rotational symmetry, and continuous rotational symmetry. Rotations are shown in 2D (Mollweide projection), with color indicating tilt direction and marker size representing pose probability murphy2021implicit. The circles represent ground truth rotations.Our correspondence-based method achieves state-of-the-art results compared to direct methods (SpyroPose and LiePose).
• Figure 2: Overview of Corr2Distrib. We first learn offline a Discrete Symmetry-Aware object model (Section \ref{['sec:learn_symmetry_aware_rep']}), consisting in a descriptor and a local frame per object surface point. At inference, given an image crop, we estimate for each pixel a descriptor and a local frame orientation relative to the camera. Then, we establish many-to-many 2D-3D correspondences between image pixels and symmetry-aware model points. From each correspondence, we estimate a 3D rotation, resulting in a set of rotation hypotheses (Section \ref{['sec:equi_pose_hypothesis']}). These rotations are filtered and leveraged to group geometrically consistent correspondences. Finally, we estimate a 6D pose from each group of correspondences and score it to generate the final object pose distribution (Section \ref{['sec:poseDistribEstim']}). Ground truth rotations appear as colored circles, and translation as a black star.
• Figure 3: Visualization of the transformations used in local frames learning. The different frames are: camera frame, object frame, and local frames (L$_X$). For a given 2D-3D correspondence $(x,X)$, $\psi_{LF}$ returns $R_{C \leftarrow L_X}$ and $\phi_{LF}$ returns $R_{L_X \leftarrow O}$. The combination of these rotations provides the desired rotation $R_{C \leftarrow O}$, which is supervised by $R^{gt}_{C \leftarrow O}$.
• Figure 4: Evolution of the rotation distribution from coarse to fine. We start by estimating an initial set of rotation hypotheses from single 2D-3D correspondences (Section \ref{['sec:equi_pose_hypothesis']}). This set is then filtered by discarding rotations according to observation conditions and $\mathrm{SO}(3)$ density regions (Section \ref{['sec:rotation_refinement']}). Finally, we refine the distribution using PnP-RANSAC and pose scoring (section \ref{['sec:poseDistribEstim']}).
• Figure 5: Failure cases. The figure illustrates two key sources of error: too small disambiguating element (first row), and texture inconsistencies between 3D object model and observed object (second row).