The Unreasonable Effectiveness of Discrete-Time Gaussian Process Mixtures for Robot Policy Learning

TL;DR

Mixture of Discrete-time Gaussian Processes (MiDiGap), a novel approach for flexible policy representation and imitation learning in robot manipulation, achieves state-of-the-art performance on diverse few-shot manipulation benchmarks.

Abstract

We present Mixture of Discrete-time Gaussian Processes (MiDiGap), a novel approach for flexible policy representation and imitation learning in robot manipulation. MiDiGap enables learning from as few as five demonstrations using only camera observations and generalizes across a wide range of challenging tasks. It excels at long-horizon behaviors such as making coffee, highly constrained motions such as opening doors, dynamic actions such as scooping with a spatula, and multimodal tasks such as hanging a mug. MiDiGap learns these tasks on a CPU in less than a minute and scales linearly to large datasets. We also develop a rich suite of tools for inference-time steering using evidence such as collision signals and robot kinematic constraints. This steering enables novel generalization capabilities, including obstacle avoidance and cross-embodiment policy transfer. MiDiGap achieves state-of-the-art performance on diverse few-shot manipulation benchmarks. On constrained RLBench tasks, it improves policy success by 76 percentage points and reduces trajectory cost by 67%. On multimodal tasks, it improves policy success by 48 percentage points and increases sample efficiency by a factor of 20. In cross-embodiment transfer, it more than doubles policy success. We make the code publicly available at https://midigap.cs.uni-freiburg.de.

Figures (17)

• Figure 1: Mixtures of Discrete-Time Gaussian Processes (MiDiGaP) effectively model multimodal trajectory distributions. It makes minimal assumptions, allowing it to model even highly-constrained movements. MiDiGaP is computationally inexpensive and scales linearly with large datasets and long horizons. Its interpretability makes it safe to execute dynamic tasks such as scooping. We develop a rich tool set to update MiDiGaP using additional evidence such as reachability and collision information.
• Figure 2: Task-parameterized policies for the OpenMicrowave task depicted in Fig. \ref{['fig:uni_tasks']}. For lucidity, we only plot the end-effector position. Subfig. (\ref{['fig:tp_gp']}) shows the local per-frame models for a discrete-time Gaussian Process. When the end-effector approaches the microwave, both the initial end-effector pose and the microwave's pose inform the trajectory. Afterwards, when grasping the handle and opening the microwave, only the microwave's pose determines the movement. TAPAS vonhartz2024art automatically finds these skill borders (vertical lines) and selects the relevant task parameters. For a specific task instance, we transform these local models into the world frame and combine them using the product of Gaussians. This process yields the joint model and prediction shown in Subfig. (\ref{['fig:tp_gp_pred']}). Subfig. (\ref{['fig:tp_gmm_pred']}) plots a GMM's prediction for the same task instance. Note how the Gaussian Process predicts a smooth trajectory suited for opening the microwave's door, whereas the GMM does not.
• Figure 3: Limitations of Continuous Gaussian Processes (CoGap) and Gaussian Mixture Models (GMM). Shaded areas indicate the 95% confidence intervals. The continuous GP and Stochastic Variational Gaussian Process (SV CoGap ) were fitted with the RBF kernel and a manually tuned length scale of $l=0.1$. Although the Matérn kernel yields similar results on the presented functions, Gaussian Mixture Models tend to smooth the trajectories between components, and the model quality is highly dependent on its initialization. Here, we used a time-based initialization scheme vonhartz2024art and manually tuned the number of parameters. Kernel-based or continuous Gaussian Processes are well-suited for simple smooth functions, but struggle with periodicity (oscillation), non-stationarity, chaotic functions, and piecewise (linear) functions. In contrast, Discrete-time Gaussian Processes (DiGaP) can model all these function classes.
• Figure 4: Modal partitioning of the mug placing skill from the multimodal PlaceCups task. We plot two dimensions of the end-effector pose over time for lucidity: the $y$ position and the $qx$ part of the quaternion. Note how the different dimensions complement one another in differentiating the modes. For example, modes 1 and 3 are difficult to distinguish in $qx$, but they can be separated more easily in $y$.
• Figure 5: A sequence $\{\mathcal{G\mkern-2muP\mkern-2muM}_1, \mathcal{G\mkern-2muP\mkern-2muM}_2, \mathcal{G\mkern-2muP\mkern-2muM}_3\}$ of three Gaussian Process Mixtures for solving the multimodal PlaceCups tasks. Arrows indicate the modes of the GPMs, nodes their start and end points. An arrow's or node's thickness indicates its likelihood under the sequence of GPMs. $\mathcal{G\mkern-2muP\mkern-2muM}_1$ is unimodal, i.e. has prior $\pi_1(m_1^1)=1$. $\mathcal{G\mkern-2muP\mkern-2muM}_2$ has three modes $m_1^2, m_2^2, m3_2$, each with prior $\frac{1}{3}$. Therefore, sequencing yields $\pi_2(1,j)=\frac{1}{3}$ for all $j\in\{1,2,3\}$. $\mathcal{G\mkern-2muP\mkern-2muM}_3$ has three modes as well. Here, we have $\pi_3(j,j)=1$ for all $j\in\{1,2,3\}$ and $\pi_3(i,j)=0$ for all $i\neq j$.