Exploring Pose-Guided Imitation Learning for Robotic Precise Insertion

TL;DR

The paper addresses the problem of robust, data-efficient robotic precise insertion under pose disturbances by introducing pose-guided imitation learning based on $SE(3)$ pose. It presents two methods: a Pose-Only Diffusion Policy that directly models future relative poses in $SE(3)^h$ using a disentangled pose encoder, and a Gated RGBD-Augmented Pose Diffusion Policy that fuses pose features with a goal-conditioned RGBD encoder via a pose-guided residual gated fusion. Key contributions include a disentangled pose encoder, a goal-conditioned RGBD image encoder, and a pose-guided fusion mechanism, achieving strong generalization across six tasks with only 7–10 demonstrations and achieving insertions with approximately $0.01$ mm clearance, outperforming several baselines. The work demonstrates data-efficient, robust, object-centric imitation learning for contact-rich manipulation and highlights future directions for integrating tactile/force feedback to further enhance performance and reliability.

Abstract

Recent studies have proved that imitation learning shows strong potential in the field of robotic manipulation. However, existing methods still struggle with precision manipulation task and rely on inefficient image/point cloud observations. In this paper, we explore to introduce SE(3) object pose into imitation learning and propose the pose-guided efficient imitation learning methods for robotic precise insertion task. First, we propose a precise insertion diffusion policy which utilizes the relative SE(3) pose as the observation-action pair. The policy models the source object SE(3) pose trajectory relative to the target object. Second, we explore to introduce the RGBD data to the pose-guided diffusion policy. Specifically, we design a goal-conditioned RGBD encoder to capture the discrepancy between the current state and the goal state. In addition, a pose-guided residual gated fusion method is proposed, which takes pose features as the backbone, and the RGBD features selectively compensate for pose feature deficiencies through an adaptive gating mechanism. Our methods are evaluated on 6 robotic precise insertion tasks, demonstrating competitive performance with only 7-10 demonstrations. Experiments demonstrate that the proposed methods can successfully complete precision insertion tasks with a clearance of about 0.01 mm. Experimental results highlight its superior efficiency and generalization capability compared to existing baselines. Code will be available at https://github.com/sunhan1997/PoseInsert.

Figures (8)

• Figure 1: As depicted in (a), RL Methods face challenges of inefficient exploration and sim2real gap. Recent studies in (b) utilize the image/point cloud as input to learn action, which can't address the precise manipulation with few demonstrations. In contrast, our framework in (c), achieves precise manipulation with few demonstrations.
• Figure 2: Human demonstration data collection. (a) Data collection and pose trajectory visualization for metal part insertion task. (b) Data collection and pose trajectory visualization for USB (Type-C) insertion task. (Trajectory Unit: Millimeter)
• Figure 3: Pipeline of the pose-guided imitation learning methods.(a) For the pose-only diffusion policy, current source object pose relative to the target object $\left \{T_{t}^{s} \right \}_{t}$ is considered as observation. The disentangled pose encoder is applied to extract the pose features. Then, diffusion policy predicts the future relative $SE(3)$ trajectory $\left \{T_{t}^{s} \right \}_{t+1:t+h}$. (b) For the gated RGBD-augmented pose diffusion policy, current relative object pose $\left \{T_{t}^{s} \right \}_{t}$ is fed to the disentangled pose encoder to get the $f_{pose}$, the current RGBD image patch is sent to goal-conditional RGBD image encoder to obtain the $f_{img}$. And the residual gated fusion module uses image features to assist pose features. The enhanced fusion feature $f_{fusion}$ is fed to diffusion policy to predict the future relative $SE(3)$ trajectory $\left \{T_{t}^{s} \right \}_{t+1:t+h}$.
• Figure 4: (a) Goal-conditioned RGBD image encoder. We employ the wen2024foundationpose's refinenet to extract the RGBD features. The two branch inputs of the refinenet are changed to the current RGBD image patch and the goal RGBD image patch, respectively. (b) Pose-guided residual gated fusion module. The MLP layer consists of a linear transformation, LayerNorm, and ReLU activation.
• Figure 5: (a) Cobot Mobile ALOHA for experiments. (b) There are $6$ precise insertion tasks for the real-world experiments.