Robot Learning Using Multi-Coordinate Elastic Maps

TL;DR

This work introduces Multi-Coordinate Elastic Maps (MC-Elmap), an extension of Elastic Maps that encodes robot demonstrations in multiple differential coordinate frames (Cartesian, Tangent, Laplacian) and automatically balances their importance during trajectory reproduction. It combines an EM-based clustering routine with a joint energy minimization over $u_\mathcal{X}$, $u_E$, $u_R$, $u_\mathcal{T}$, and $u_\mathcal{L}$, accompanied by meta-optimization for hyperparameters and online update capability. The method is validated on 2D LASA handwriting, 3D robotic tasks (RAIL), and a real-world UR5e handwriting task, showing smoother, shape-preserving reproductions and superior metrics (e.g., Fréchet distance, SSE, angular similarity, jerk) compared to baselines. The results demonstrate MC-Elmap's ability to uncover and reproduce task-relevant features beyond Cartesian space, offering practical benefits for robust robot manipulation and handwriting-like skills.

Abstract

To learn manipulation skills, robots need to understand the features of those skills. An easy way for robots to learn is through Learning from Demonstration (LfD), where the robot learns a skill from an expert demonstrator. While the main features of a skill might be captured in one differential coordinate (i.e., Cartesian), they could have meaning in other coordinates. For example, an important feature of a skill may be its shape or velocity profile, which are difficult to discover in Cartesian differential coordinate. In this work, we present a method which enables robots to learn skills from human demonstrations via encoding these skills into various differential coordinates, then determines the importance of each coordinate to reproduce the skill. We also introduce a modified form of Elastic Maps that includes multiple differential coordinates, combining statistical modeling of skills in these differential coordinate spaces. Elastic Maps, which are flexible and fast to compute, allow for the incorporation of several different types of constraints and the use of any number of demonstrations. Additionally, we propose methods for auto-tuning several parameters associated with the modified Elastic Map formulation. We validate our approach in several simulated experiments and a real-world writing task with a UR5e manipulator arm.

Figures (6)

• Figure 1: Multi-Coordinate Elastic Maps (MC-Elmap) performed on a real-world writing skill. Multiple "R" shapes are drawn using a real-world UR5e manipulator arm, but the demonstrations are of low quality as it is difficult to apply enough pressure to write the shape while drawing smoothly. However, MC-Elmap is able to successfully and smoothly reproduce the skill, emphasizing features which were not present in the demonstrations.
• Figure 2: Results of performing MC-Elmap (red) on the handwriting shapes LASA dataset Khansari-Zadeh2011LASA (demonstrations shown in gray).
• Figure 3: Boxplots comparing the results of measuring the Fréchet distance, Sum of Squared Errors (SSE), Angular Similarity, and Jerk for different LfD representations on the LASA Dataset (see Fig. \ref{['fig:lasa']}). MC-Elmap (denoted MC-E in the plots) performs well for all metrics. Boxplot whiskers show 1.5 interquartile range (IQR), with the median shown in red.
• Figure 4: A comparison between uniform weighting (shown in green) and automatic weighting (shown in red) for MC-Elmap on the "NShape" of the LASA dataset (demonstrations shown in gray). Uniform weighting places a higher importance on Cartesian similarity, leading to some jaggedness the reproduction in order to increase spatial similarity in the reproduction, whereas the automatic weighting places a higher importance on Laplacian similarity, maintaining the shape of the handwriting skill.
• Figure 5: Results of performing MC-Elmap (red) on the RAIL dataset rana2020benchmark (demonstrations in gray). A sample real-world demonstration of each skill is shown in the bottom corner for each skill. MC-Elmap is able to successfully reproduce all skills from multiple different starting points, and even is able to reproduce features without constraints, such as the first press in the pressing task.