M2Distill: Multi-Modal Distillation for Lifelong Imitation Learning

TL;DR

LIL for robotic manipulation suffers from distribution shifts that cause latent-space drift and forgetting. M2Distill introduces a dual-distillation framework: (1) multi-modal latent-space distillation that aligns vision, language, and action embeddings across incremental steps via L2 losses, and (2) policy distillation that preserves the previous GMM action distribution using KL divergence with Monte Carlo approximation. The approach is instantiated on a ResNet-T backbone with language embeddings and multi-view encoders, and evaluated on LIBERO-OBJECT, LIBERO-GOAL, and LIBERO-SPATIAL, where it outperforms baselines across FWT, NBT, and AUC, with ablations confirming the necessity of each component. The results demonstrate improved stability and continual learning capability for real-world manipulation tasks, suggesting practical impact for scalable, continual robotic learning. Key mathematical constructs include J(π) = (1/K) Σ_k E_{s_t^k,a_t^k ~ π(·;T^k),μ_0^k}[Σ_t g^k(s_t^k)] and distillation losses L_image, L_text, L_extra, and L_policy with L2 and KL terms, respectively.

Abstract

Lifelong imitation learning for manipulation tasks poses significant challenges due to distribution shifts that occur in incremental learning steps. Existing methods often focus on unsupervised skill discovery to construct an ever-growing skill library or distillation from multiple policies, which can lead to scalability issues as diverse manipulation tasks are continually introduced and may fail to ensure a consistent latent space throughout the learning process, leading to catastrophic forgetting of previously learned skills. In this paper, we introduce M2Distill, a multi-modal distillation-based method for lifelong imitation learning focusing on preserving consistent latent space across vision, language, and action distributions throughout the learning process. By regulating the shifts in latent representations across different modalities from previous to current steps, and reducing discrepancies in Gaussian Mixture Model (GMM) policies between consecutive learning steps, we ensure that the learned policy retains its ability to perform previously learned tasks while seamlessly integrating new skills. Extensive evaluations on the LIBERO lifelong imitation learning benchmark suites, including LIBERO-OBJECT, LIBERO-GOAL, and LIBERO-SPATIAL, demonstrate that our method consistently outperforms prior state-of-the-art methods across all evaluated metrics.

Figures (4)

• Figure 1: t-SNE visualization of latent space deformation for AgentView images using Experience Replay (ER) method across two consecutive steps (i.e., $T^2$ and $T^3$) in a lifelong imitation learning scenario on LIBERO-OBJECT. The t-SNE plots highlight significant shifts in latent representations, contributing to catastrophic forgetting, while the accompanying bar plot shows reduced representation drift in M2Distill during the sequential learning of manipulation tasks.
• Figure 2: Overview of our proposed M2Distill method. Multi-modal distillation aligns the latent representations from different input modality encoders (e.g., Task, AgentView, Eye-in-Hand, Joint, and Gripper information), while policy distillation maps the action distribution of the GMM policy between incremental steps $T^{k-1}$ and $T^{k}$.
• Figure 3: The success rate across the incremental steps for ER and M2Distill (Ours) on LIBEO-OBJECT task suite. Our method demonstrates a more consistent success rate across incremental steps compared to the ER baseline method. Higher values indicate better performance.
• Figure 4: Latent representation drift across the incremental steps for ER and M2Distill (Ours) on LIBEO-OBJECT task suite. Drift is measured using the squared Euclidean distance between latent representations from policies at $t$ and $t-1$. Our method maintains a more consistent latent space across modalities compared to the ER baseline method. Lower values indicate better performance.