Guided Decoding for Robot On-line Motion Generation and Adaption

TL;DR

This work trains a transformer architecture, based on conditional variational autoencoder, on a large dataset of simulated trajectories used as demonstrations, that learns essential motion generation skills from these demonstrations and is able to adapt them to meet auxiliary tasks.

Abstract

We present a novel motion generation approach for robot arms, with high degrees of freedom, in complex settings that can adapt online to obstacles or new via points. Learning from Demonstration facilitates rapid adaptation to new tasks and optimizes the utilization of accumulated expertise by allowing robots to learn and generalize from demonstrated trajectories. We train a transformer architecture, based on conditional variational autoencoder, on a large dataset of simulated trajectories used as demonstrations. Our architecture learns essential motion generation skills from these demonstrations and is able to adapt them to meet auxiliary tasks. Additionally, our approach implements auto-regressive motion generation to enable real-time adaptations, as, for example, introducing or changing via-points, and velocity and acceleration constraints. Using beam search, we present a method for further adaption of our motion generator to avoid obstacles. We show that our model successfully generates motion from different initial and target points and that is capable of generating trajectories that navigate complex tasks across different robotic platforms.

Figures (7)

• Figure 1: Model overview. Each PerceiverIO module gets a trajectory, but for the prior it only contains the beginning and end points and a full trajectory for the posterior. During training, minimizing the KL divergence between the two makes the representation of the PerceiverIO module that only gets $x_1$ and $x_T$ generates the same latent distribution as the one getting the full trajectory; therefore, it learns interpolation. The latent state then is used by the autoregressive decoder to generate the next setpoint $x_{t+1}$. Dashed lines indicate components used exclusively during the post-training adaptation phase, not included in the initial training process.
• Figure 2: Distances to the original trajectories for the Panda robot in Cartesian space.
• Figure 3: Distances to the original trajectories for IIWA robot in joint space.
• Figure 4: Distances to the original trajectories for the Panda robot in Cartesian space with multiple-robot training and single-robot training.
• Figure 5: Motion adaptation for position boundary. Colors indicate the axes of the Cartesian coordinate.