Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TransPose: 6D Object Pose Estimation with Geometry-Aware Transformer

Xiao Lin, Deming Wang, Guangliang Zhou, Chengju Liu, Qijun Chen

TL;DR

TransPose addresses 6D object pose estimation from depth-derived point clouds by integrating a graph-based local feature extractor with a geometry-aware Transformer Encoder. The geometry-aware module provides an inductive bias that couples global information exchange to the local geometry of the point cloud, enabling robust performance under occlusion. The method achieves competitive results on LineMod, Occlusion LineMod, and YCB-Video without relying on RGB inputs or synthetic refinements, underscoring the efficacy of geometry-guided global fusion for 3D pose tasks. This approach advances depth-only pose estimation by effectively combining local geometric structure with global contextual reasoning, offering practical impact for robotics and AR applications where depth data is available but texture is limited.

Abstract

Estimating the 6D object pose is an essential task in many applications. Due to the lack of depth information, existing RGB-based methods are sensitive to occlusion and illumination changes. How to extract and utilize the geometry features in depth information is crucial to achieve accurate predictions. To this end, we propose TransPose, a novel 6D pose framework that exploits Transformer Encoder with geometry-aware module to develop better learning of point cloud feature representations. Specifically, we first uniformly sample point cloud and extract local geometry features with the designed local feature extractor base on graph convolution network. To improve robustness to occlusion, we adopt Transformer to perform the exchange of global information, making each local feature contains global information. Finally, we introduce geometry-aware module in Transformer Encoder, which to form an effective constrain for point cloud feature learning and makes the global information exchange more tightly coupled with point cloud tasks. Extensive experiments indicate the effectiveness of TransPose, our pose estimation pipeline achieves competitive results on three benchmark datasets.

TransPose: 6D Object Pose Estimation with Geometry-Aware Transformer

TL;DR

TransPose addresses 6D object pose estimation from depth-derived point clouds by integrating a graph-based local feature extractor with a geometry-aware Transformer Encoder. The geometry-aware module provides an inductive bias that couples global information exchange to the local geometry of the point cloud, enabling robust performance under occlusion. The method achieves competitive results on LineMod, Occlusion LineMod, and YCB-Video without relying on RGB inputs or synthetic refinements, underscoring the efficacy of geometry-guided global fusion for 3D pose tasks. This approach advances depth-only pose estimation by effectively combining local geometric structure with global contextual reasoning, offering practical impact for robotics and AR applications where depth data is available but texture is limited.

Abstract

Estimating the 6D object pose is an essential task in many applications. Due to the lack of depth information, existing RGB-based methods are sensitive to occlusion and illumination changes. How to extract and utilize the geometry features in depth information is crucial to achieve accurate predictions. To this end, we propose TransPose, a novel 6D pose framework that exploits Transformer Encoder with geometry-aware module to develop better learning of point cloud feature representations. Specifically, we first uniformly sample point cloud and extract local geometry features with the designed local feature extractor base on graph convolution network. To improve robustness to occlusion, we adopt Transformer to perform the exchange of global information, making each local feature contains global information. Finally, we introduce geometry-aware module in Transformer Encoder, which to form an effective constrain for point cloud feature learning and makes the global information exchange more tightly coupled with point cloud tasks. Extensive experiments indicate the effectiveness of TransPose, our pose estimation pipeline achieves competitive results on three benchmark datasets.
Paper Structure (16 sections, 7 equations, 5 figures, 7 tables)

This paper contains 16 sections, 7 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. 6D Object Pose Estimation
  4. Graph Convolution Network (GCN)
  5. Vision Transformer
  6. Proposed Method
  7. Overview
  8. Point Cloud Feature Embeddings Generation
  9. Geometry-aware Transformer
  10. Pose Prediction Network
  11. Experiments
  12. Datasets and Metrics
  13. Ablation Studies
  14. Comparison with the Same Type Methods
  15. Evaluations on Benchmark Datasets
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Illustration of TransPose. Given point cloud of objects as input, the model uniformly samples several local regions of the point cloud and extracts local neighborhood features via local feature extractor base on graph convolution network. The obtained feature form a point cloud embeddings, which is fed to a transformer encoder with the geometry-aware module to obtain the global features. Finally, the pose estimation network recovers object 6D pose parameters.
  • Figure 2: Overview of TransPose. The pre-processing stage obtaining the target object point cloud from the mask and depth image of the object with the camera internal reference transform(e.g., cat). The model employs a GCN-based feature extractor to obtain a local feature representation of the point cloud, and supplements it with a learnable positional encoding before passing into the Transformer Encoder. Transformer block takes as input a local feature embeddings then fuses the results of the multi-head attention and geometry-aware module to produce a global feature representation The ultimate 6D pose estimation parameters are recovered by Pose Prediction Network.
  • Figure 3: The framework of GCN-based Local Feature Extractor. The main network is composed of two modules: (1)Graph Convolution, which is the key component for extraction of local features. The module conducts K-Nearest Neighbor(K-NN) to determine the topology of the graph structure and converges neighborhood information to local centers via pooling. (2)Furthest Point Sampling (FPS), which exploits downsampling to reduce the number of point cloud sub-regions. The two modules connected at the string level are able to extract robust local features while boosting the efficiency of the algorithm.
  • Figure 4: Pose Prediction Network. The network predicts the translation and rotation components through two decoupled branches respectively. Both sub-networks consist of a cascade of 1×1 convolution modules.
  • Figure 5: Qualitative results on the YCB-Video dataset. We project the predicted poses as point cloud onto each model in the RGB image. Different objects are depicted by different colors.