APEX: Ambidextrous Dual-Arm Robotic Manipulation Using Collision-Free Generative Diffusion Models

TL;DR

APEX introduces collision-free ambidextrous dual-arm manipulation via latent diffusion models that learn diverse, safe manipulation trajectories while handling dynamic obstacles. Tasks are distilled into vector alignment problems and learned in a latent space with a VAE, using classifier-guided diffusion to incorporate real-time obstacle information and close the loop. Hardware experiments on two Franka Panda arms with depth sensing validate robust, smooth, and diverse trajectories, outperforming RRT, GPMP, IL-based methods, and prior diffusion approaches in both failure rate and smoothness. The approach promises scalable, efficient planning for complex manipulation tasks in real-world, dynamic environments and opens avenues for diffusion-based control in other robotic platforms.

Abstract

Dexterous manipulation, particularly adept coordinating and grasping, constitutes a fundamental and indispensable capability for robots, facilitating the emulation of human-like behaviors. Integrating this capability into robots empowers them to supplement and even supplant humans in undertaking increasingly intricate tasks in both daily life and industrial settings. Unfortunately, contemporary methodologies encounter serious challenges in devising manipulation trajectories owing to the intricacies of tasks, the expansive robotic manipulation space, and dynamic obstacles. We propose a novel approach, APEX, to address all these difficulties by introducing a collision-free latent diffusion model for both robotic motion planning and manipulation. Firstly, we simplify the complexity of real-life ambidextrous dual-arm robotic manipulation tasks by abstracting them as aligning two vectors. Secondly, we devise latent diffusion models to produce a variety of robotic manipulation trajectories. Furthermore, we integrate obstacle information utilizing a classifier-guidance technique, thereby guaranteeing both the feasibility and safety of the generated manipulation trajectories. Lastly, we validate our proposed algorithm through extensive experiments conducted on the hardware platform of ambidextrous dual-arm robots. Our algorithm consistently generates successful and seamless trajectories across diverse tasks, surpassing conventional robotic motion planning algorithms. These results carry significant implications for the future design of diffusion robots, enhancing their capability to tackle more intricate robotic manipulation tasks with increased efficiency and safety. Complete video demonstrations of our experiments can be found in https://sites.google.com/view/apex-dual-arm/home.

Figures (10)

• Figure 1: a)Dual-arm robotic manipulation platform (b) Overall diagram of our algorithm. (c) Our diffusion models learn the distribution of trajectory using training data (black line) and generate a new planning trajectory (green dashed line). Further, it adapts the generated trajectory (red dashed line) to avoid the potential obstacles (red box).
• Figure 2: Solving the real-world dexterous dual-arm robotic manipulation tasks. The first row indicates various robotic manipulation tasks. The second row is the initial position of the manipulation tasks. The third row distills the manipulation tasks by abstracting them as vector alignment problems, which can be solved using our proposed algorithm. The last row shows the successful completeness of the manipulation tasks.
• Figure 3: The diagram of our ambidextrous dual-arm robotic manipulation algorithm using latent diffusion models to generate the collision-free trajectory. The input of our trajectory data $x$ first goes into a pre-trained variational autoencoder to have a low-dimensional latent embedding $z_0$, which will be used to train a diffusion model. In the sampling procedure, we randomly generate Gaussian noise trajectory $z_T$ and pass it through reverse denoising steps to generate clean trajectory latent embedding $z^{'}_0$, which can be decoded using a pre-trained VAE decoder. Further, to ensure the generated trajectory $x^{'}$ is collusion-free, we add two mechanisms on top of the latent diffusion models. First, we use the obstacle information $g$ as the conditional constraint in both training and sampling process. Second, our trajectory generation process works in a close-loop manner, where we check the possibility of collision violation and resample a new trajectory if necessary.
• Figure 4: Snapshots of hardware demonstration from vertical (a)-(e) and horizontal alignment task (f)-(j). (a)-(b) Robotic arms grasp the boxes from a table. (c)-(d) Right robotic arm avoids possible collision (red box). (e) Successful task completed. (f)-(g) Two arms take the cup and the ball to prepare a task in hand. (h) A dynamic obstacle appears in the workspace and the left robotic arm faces probable collision, (i) The Left arm moves over the obstacle to avoid collision. Now, the right arm gets into a probable collision with the moving red box. (j) Both arms dodge the obstacles and complete the task.
• Figure 5: Denoising diffusion process for the generation of manipulation trajectories. From left to right, we gradually show how the denoising diffusion process generates a smooth and feasible trajectory to reach the predefined goal position.