SEMPose: A Single End-to-end Network for Multi-object Pose Estimation

TL;DR

SEMPose, an end-to-end multi-object pose estimation network that utilizes a well-designed texture-shape guided feature pyramid network, effectively tackling the challenge of object size variations, outperforms other RGB-based single-model methods, achieving higher accuracy.

Abstract

In computer vision, estimating the six-degree-of-freedom pose from an RGB image is a fundamental task. However, this task becomes highly challenging in multi-object scenes. Currently, the best methods typically employ an indirect strategy, which identifies 2D and 3D correspondences, and then solves with the Perspective-n-Points method. Yet, this approach cannot be trained end-to-end. Direct methods, on the other hand, suffer from lower accuracy due to challenges such as varying object sizes and occlusions. To address these issues, we propose SEMPose, an end-to-end multi-object pose estimation network. SEMPose utilizes a well-designed texture-shape guided feature pyramid network, effectively tackling the challenge of object size variations. Additionally, it employs an iterative refinement head structure, progressively regressing rotation and translation separately to enhance estimation accuracy. During training, we alleviate the impact of occlusion by selecting positive samples from visible parts. Experimental results demonstrate that SEMPose can perform inference at 32 FPS without requiring inputs other than the RGB image. It can accurately estimate the poses of multiple objects in real time, with inference time unaffected by the number of target objects. On the LM-O and YCB-V datasets, our method outperforms other RGB-based single-model methods, achieving higher accuracy. Even when compared with multi-model methods and approaches that use additional refinement, our results remain competitive.

Figures (7)

• Figure 1: Framework of SEMPose. Given an input RGB image I, SEMPose first uses Resnet50 to extract basic features. Then, SEMPose employs a Texture-Shape Guided Feature Pyramid Network (TS-FPN) to fuse these features and produce feature maps at five different scales. Finally, SEMPose uses four heads to predict the categories, bounding boxes, rotations, and translations of the objects. After post-processing, the poses are obtained. Additionally, the network's training process relies on a positive sample sampling strategy guided by the visible parts.
• Figure 2: The structure diagram of texture shape feature fusion module.
• Figure 3: The structure of the rotation head.
• Figure 4: Schematic diagrams of the three different sampling strategies tian2022fcoszhang2020atsshai2023rigidity. A darker green color indicates a higher probability of sampling.
• Figure 5: The schematic diagram of the sampling process from visible parts.