SD-Net: Symmetric-Aware Keypoint Prediction and Domain Adaptation for 6D Pose Estimation In Bin-picking Scenarios

TL;DR

A novel 6D pose estimation network with symmetric-aware keypoint prediction and self-training domain adaptation (SD-Net) and a robust 3D keypoint selection strategy considering the symmetry class of objects and equivalent keypoints, which facilitate locating 3D keypoints even in highly occluded scenes.

Abstract

Despite the success in 6D pose estimation in bin-picking scenarios, existing methods still struggle to produce accurate prediction results for symmetry objects and real world scenarios. The primary bottlenecks include 1) the ambiguity keypoints caused by object symmetries; 2) the domain gap between real and synthetic data. To circumvent these problem, we propose a new 6D pose estimation network with symmetric-aware keypoint prediction and self-training domain adaptation (SD-Net). SD-Net builds on pointwise keypoint regression and deep hough voting to perform reliable detection keypoint under clutter and occlusion. Specifically, at the keypoint prediction stage, we designe a robust 3D keypoints selection strategy considering the symmetry class of objects and equivalent keypoints, which facilitate locating 3D keypoints even in highly occluded scenes. Additionally, we build an effective filtering algorithm on predicted keypoint to dynamically eliminate multiple ambiguity and outlier keypoint candidates. At the domain adaptation stage, we propose the self-training framework using a student-teacher training scheme. To carefully distinguish reliable predictions, we harnesses a tailored heuristics for 3D geometry pseudo labelling based on semi-chamfer distance. On public Sil'eane dataset, SD-Net achieves state-of-the-art results, obtaining an average precision of 96%. Testing learning and generalization abilities on public Parametric datasets, SD-Net is 8% higher than the state-of-the-art method. The code is available at https://github.com/dingthuang/SD-Net.

Figures (7)

• Figure 1: This paper addresses two major problems facing 6D pose estimation. (a) Keypoints sampled by BBox, FPS, Parameter and ours strategy. BBox, FPS and Parameter keypoints ignore object geometric symmetry characteristics, while ours strategy adaptively select keypoints based on object symmetry class. (b) Methods typically exhibit inferior performance when applied to real world point clouds, due to the persistent domain gap between real and synthetic data point clouds.
• Figure 2: Overview of SD-Net architecture. SD-Net is constructed based on point-wise keypoint regression and deep Hough voting. It consists of two main parts: symmetri-caware keypoint prediction and self-supervised domain adaptation. keypoint prediction consists of a new keypoint selection and filtering algorithm. We omit the domain adaptation framework, for brevity and more details can be found in Section \ref{['subsec: domain']}. $N_j$ represents the number of point cloud points for each instance. $N_k$ represents the number of keypoint decoders and corresponds to the number of keypoint of objects. $N_i$ represents the number of instances in the scene.
• Figure 3: The axes selection strategy depends on the object symmetry class. The red axes represent the selected axes. The blue dots represent the selected keypoints, while the red dots represent the corresponding equivalent keypoints.
• Figure 4: (a) Keypoints sampled by ours selection algorithms on t-less20 object from Siléane dataset. (b) The white points represent the scene instance point clouds, and the red points represent the pointwise predicted keypoints. (c) The red points represent predicted keypoints after filtering.
• Figure 5: Overview of our proposed self-supervised domain adaptation framework for 6D object pose estimation. We first train teacher model on synthesize abundant data to generate initial pose predictions. We then use a 3D geometry pseudo labelling algorithm to distinguish real word predictions for student model training. In the next iteration, the teacher model is initialized as the last trained student model and iterate the above process util the model convergence.