One-shot Imitation in a Non-Stationary Environment via Multi-Modal Skill

TL;DR

The paper tackles one-shot imitation for long-horizon tasks under non-stationary dynamics by decomposing behavior into semantic skills and environment dynamics. It introduces OnIS, a framework that leverages a vision-language embedding (CLIP) to represent semantic skills and a dynamics encoder with vector quantization to enable dynamics-aware skill transfer, trained with contrastive and reconstruction objectives. During deployment, a single demonstration yields a semantic skill sequence that is rapidly adapted to current dynamics, enabling zero-shot adaptation to time-varying environments. Experiments on multi-stage Meta-World tasks demonstrate that OnIS outperforms baselines across video and language modalities, with robustness to noise and unseen dynamics, illustrating the practical potential of multi-modal, compositional imitation in non-stationary settings.

Abstract

One-shot imitation is to learn a new task from a single demonstration, yet it is a challenging problem to adopt it for complex tasks with the high domain diversity inherent in a non-stationary environment. To tackle the problem, we explore the compositionality of complex tasks, and present a novel skill-based imitation learning framework enabling one-shot imitation and zero-shot adaptation; from a single demonstration for a complex unseen task, a semantic skill sequence is inferred and then each skill in the sequence is converted into an action sequence optimized for environmental hidden dynamics that can vary over time. Specifically, we leverage a vision-language model to learn a semantic skill set from offline video datasets, where each skill is represented on the vision-language embedding space, and adapt meta-learning with dynamics inference to enable zero-shot skill adaptation. We evaluate our framework with various one-shot imitation scenarios for extended multi-stage Meta-world tasks, showing its superiority in learning complex tasks, generalizing to dynamics changes, and extending to different demonstration conditions and modalities, compared to other baselines.

Figures (9)

• Figure 1: Multi-stage Meta-world task
• Figure 2: One-shot imitation and zero-shot adaptation in OnIS: (a) In the training phase, a semantic skill set is established from video demonstrations in offline datasets by leveraging a pretrained vision-language model. For each demonstration in the dataset, semantic skill sequence and environment dynamics are first disentangled, and then they are combined to enable dynamics-aware skill transfer to different environments. (b) In the deployment phase, given a single demonstration for an unseen task, its semantic skill sequence can be immediately inferred (one-shot imitation), where each in the sequence adapts temporally to the hidden, time-varying dynamics in the environment (zero-shot adaptation).
• Figure 3: OnIS framework: In (a), the semantic skill sequence encoder $\Phi_{enc}$ and the semantic skill decoder $\Phi_{dec}$ are trained offline using the CLIP vision-language pretrained model, where $\Phi_{enc}$ translates video demonstrations to semantic skill sequences and is contrastively learned, and $\Phi_{dec}$ learns to infer an optimal skill (from a sequence) upon a state. The language prompt $\theta_p$ is only used for the subtask-level instruction (S-OnIS) case, and the additional encoders $\theta_v, \theta_l$ are only used for episode-level instruction case (U-OnIS). In (b), the skill transfer $\pi_{tr}$ and the dynamics encoder $\psi_{enc}$ are trained offline, where $\pi_{tr}$ learns to infer an action sequence optimized for the deployment setting from a given semantic skill sequence and inferred dynamics, and $\psi_{enc}$ learns to infer dynamics from sub-trajectories. These modules establish dynamics-aware skill transfer. In (c), for a given demonstration, $\Phi_{enc}$ first infers a sequence of semantic skills, $\Phi_{dec}$ infers a current semantic skill, and $\psi_{enc}$ infers current dynamics in the non-stationary deployment environment. Then, $\pi_{tr}$ yields actions optimized through the current semantic skill and dynamics.
• Figure 4: Semantic embedding correspondence for a demonstration and one-shot imitation run
• Figure 5: Effect by annotated sample size: the x-axis denotes the number of annotated samples used for training the semantic skill sequence encoder in S-OnIS, and the y-axis denotes the one-shot imitation performance by S-OnIS.