Incremental Learning of Retrievable Skills For Efficient Continual Task Adaptation

TL;DR

This work tackles continual imitation learning (CiL) by addressing the limited knowledge sharing of existing adapter-based methods. It introduces IsCiL, which stores and incrementally learns shareable skills as prototypes and per-skill adapters, enabling retrieval-based composition of actions for new tasks and non-stationary environments. The approach uses a fixed state encoder, a prototype-based skill retriever, and a LoRA-conditioned skill decoder to maintain stability while allowing scalable adaptation; it also supports task unlearning by removing specific skill prototypes and adapters. Empirical results on Franka-Kitchen and Meta-World demonstrate improved sample efficiency and robust task adaptation across Complete, Semi-complete, and Incomplete streams, with effective handling of unseen tasks and privacy-preserving unlearning. Overall, IsCiL narrows the gap between adapter-based CiL and cross-task knowledge sharing, delivering practical benefits for continual task learning in complex, long-horizon domains.

Abstract

Continual Imitation Learning (CiL) involves extracting and accumulating task knowledge from demonstrations across multiple stages and tasks to achieve a multi-task policy. With recent advancements in foundation models, there has been a growing interest in adapter-based CiL approaches, where adapters are established parameter-efficiently for tasks newly demonstrated. While these approaches isolate parameters for specific tasks and tend to mitigate catastrophic forgetting, they limit knowledge sharing among different demonstrations. We introduce IsCiL, an adapter-based CiL framework that addresses this limitation of knowledge sharing by incrementally learning shareable skills from different demonstrations, thus enabling sample-efficient task adaptation using the skills particularly in non-stationary CiL environments. In IsCiL, demonstrations are mapped into the state embedding space, where proper skills can be retrieved upon input states through prototype-based memory. These retrievable skills are incrementally learned on their corresponding adapters. Our CiL experiments with complex tasks in Franka-Kitchen and Meta-World demonstrate robust performance of IsCiL in both task adaptation and sample-efficiency. We also show a simple extension of IsCiL for task unlearning scenarios.

Figures (8)

• Figure 1: The scenario demonstrating how IsCiL enhances continual imitation learning efficiency through retrievable skills: (i) Prototype-based skill incremental learning: despite the failure of $\tau_1$, skills are incrementally learned from the available demonstrations. In later stages, missing skills for $\tau_1$ are retrieved from other tasks, achieving the resolution of $\tau_1$ and illustrating the reversibility and efficiency of retrievable skills. (ii) Task-wise selective adaptation: IsCiL effectively retrieves relevant learned skills, facilitating rapid task adaptation.
• Figure 2: Overview of the IsCiL framework: (a) The prototype-based skill retriever sequentially utilizes a state encoder $f$, multifaceted skill prototypes $\mathcal{X}$, and a skill adapter mapping function $h$ to identify the skill adapter $\theta_{z}$. (b) Skill incremental learning involves the initialization and updating of the skill prototype $\chi_{z^*}$ and its corresponding adapter $\theta_{z^*}$.
• Figure 3: CiL scenarios including Complete, Semi-Complete, and Incomplete, categorized by sample utilization difficulty, based on the completeness of the demonstration for task performance: In Complete, each of the 20 CiL stages incrementally introduces new tasks featuring objects not encountered in the pre-training stage, along with full, comprehensive demonstrations for each task. In Semi-Complete, the first 10 stages are repeated twice, with tasks presented alongside incomplete demonstrations, where specific sub-goals are missing from the trajectories. In Incomplete, the same sequence of tasks from the Complete scenario is used, but all stages feature incomplete demonstrations, requiring the system to handle tasks with missing sub-goal trajectories.
• Figure 4: Comparison w.r.t. the number of rehearsals: The horizontal axis represents the amount of stored rehearsal data at each stage, while the vertical axis indicates goal-conditioned success rates (GC).
• Figure 5: Comparison w.r.t. training resources: In all baselines, the plain bar graph represents FWT, while the bar graph with hatch marks represents AUC. The vertical axis indicates goal-conditioned success rates (GC).