SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrations

TL;DR

SkillMimic-V2 tackles the challenge of learning robust interaction skills from sparse and noisy demonstrations in RLID. It introduces two data augmentation constructs, a Stitched Trajectory Graph (STG) and a State Transition Field (STF), to uncover feasible transitions between demonstrated skills, complemented by Adaptive Trajectory Sampling (ATS) and a History Encoder (HE) to guide memory-aware learning. The framework significantly improves convergence stability, generalization, and recovery from perturbations across BallPlay-M and ParaHome datasets, outperforming state-of-the-art baselines and showing strong cross-skill transfer with limited demonstrations. By bridging gaps in demonstration coverage and enabling targeted, memory-informed curriculum, SkillMimic-V2 offers a practical pathway to scalable, robust robot-object interaction learning with sparse data.

Abstract

We address a fundamental challenge in Reinforcement Learning from Interaction Demonstration (RLID): demonstration noise and coverage limitations. While existing data collection approaches provide valuable interaction demonstrations, they often yield sparse, disconnected, and noisy trajectories that fail to capture the full spectrum of possible skill variations and transitions. Our key insight is that despite noisy and sparse demonstrations, there exist infinite physically feasible trajectories that naturally bridge between demonstrated skills or emerge from their neighboring states, forming a continuous space of possible skill variations and transitions. Building upon this insight, we present two data augmentation techniques: a Stitched Trajectory Graph (STG) that discovers potential transitions between demonstration skills, and a State Transition Field (STF) that establishes unique connections for arbitrary states within the demonstration neighborhood. To enable effective RLID with augmented data, we develop an Adaptive Trajectory Sampling (ATS) strategy for dynamic curriculum generation and a historical encoding mechanism for memory-dependent skill learning. Our approach enables robust skill acquisition that significantly generalizes beyond the reference demonstrations. Extensive experiments across diverse interaction tasks demonstrate substantial improvements over state-of-the-art methods in terms of convergence stability, generalization capability, and recovery robustness.

Figures (7)

• Figure 1: Given a degraded reference trajectory containing physically unreachable state transitions, perfect trajectory reconstruction becomes impossible. The goal is to learn a set of ideal trajectories that are both physically feasible and satisfy reconstruction thresholds. These ideal trajectories must exist within an $\boldsymbol{\varepsilon}$-neighborhood of the reference trajectory.
• Figure 2: Given sparse demonstrations (e.g., two short trajectories of Shot and Dribble), there exist infinite valid but uncaptured trajectories that can either bridge between them or emerge from their neighboring states (illustrated by question marks). Our method uncovers these potential trajectories via three key steps: (1) construct a Stitched Trajectory Graph (STG) to identify possible transitions, (2) expand STG into a State Transition Field (STF) that establishes connections for arbitrary states within the demonstration neighborhood, and (3) learn a skill policy via Adaptive Trajectory Sampling (ATS) and Reinforcement Learning from Interaction Demonstrations (RLID). This enables robust skill transition and generalization far beyond the original sparse demonstrations.
• Figure 3: Qualitative comparison on BallPlay-M. Blue trajectories in (a,b) indicate executions beyond the reference Layup data length. In (c,d), green and blue trajectories represent dribbling left (DL) and dribbling right (DR) respectively, demonstrating skill transition not present in the reference data.
• Figure 4: Qualitative comparison on ParaHome. Humanoid performing (a,c) tea-pouring and teapot placement, (b,d) standing and chair-pushing sequences.
• Figure 5: Performance comparisons of the proposed approach against baselines across four key metrics.