Towards Robot Skill Learning and Adaptation with Gaussian Processes

TL;DR

A novel, robust skill adaptation framework that utilises GPs with sparse via-points for compact and expressive modelling and demonstrates that the GP-based representation enables all three methods to attain high cosine similarity and low velocity magnitude errors, indicating strong preservation of the kinematic profile.

Abstract

General robot skill adaptation requires expressive representations robust to varying task configurations. While recent learning-based skill adaptation methods refined via Reinforcement Learning (RL), have shown success, existing skill models often lack sufficient representational capacity for anything beyond minor environmental changes. In contrast, Gaussian Process (GP)-based skill modelling provides an expressive representation with useful analytical properties; however, adaptation of GP-based skills remains underexplored. This paper proposes a novel, robust skill adaptation framework that utilises GPs with sparse via-points for compact and expressive modelling. The model considers the trajectory's poses and leverages its first and second analytical derivatives to preserve the skill's kinematic profile. We present three adaptation methods to cater for the variability between initial and observed configurations. Firstly, an optimisation agent that adjusts the path's via-points while preserving the demonstration velocity. Second, a behaviour cloning agent trained to replicate output trajectories from the optimisation agent. Lastly, an RL agent that has learnt to modify via-points whilst maintaining the kinematic profile and enabling online capabilities. Evaluated across three tasks (drawer opening, cube-pushing and bar manipulation) in both simulation and hardware, our proposed methods outperform every benchmark in success rates. Furthermore, the results demonstrate that the GP-based representation enables all three methods to attain high cosine similarity and low velocity magnitude errors, indicating strong preservation of the kinematic profile. Overall, our formulation provides a compact representation capable of adapting to large deviations from a single demonstrated skill.

Figures (5)

• Figure 1: The agent learns to adapt the skill from a single demonstration (top) for one-shot deployment on a real robot under various initial conditions (bottom).
• Figure 2: A single demonstration is parameterised through a GP formulation with sparse via-points. The subsequent skill adaptation method learns to tune the via-points to accomplish the task in varying TCs. This can be successfully transferred to hardware for one-shot execution.
• Figure 3: Retention of demonstrated velocities: Both Skill-GP and Skill Cloning demonstrate near-perfect retention. GPRL is able to obtain competitive scores in comparison to the vanilla GP. ProMP and ProMP-RRL show low magnitude error, but with poorer directional similarities.
• Figure 4: Simulated environments for drawer pulling, cube pushing and bar manipulation tasks.
• Figure 5: Vanilla GP crashes onto the bar (left), whereas our methods successfully adapt the learnt skill (right).