COLA: Continual Learning via Autoencoder Retrieval of Adapters

TL;DR

This paper tackles continual learning in large language models under no-rehearsal memory constraints. It introduces COLA, a framework that freezes a pretrained backbone and uses per-task LoRA adapters whose weights are compressed into compact latent codes by a contractive autoencoder, enabling on-demand reconstruction during inference. Key contributions include a three-stage process (adapt, encode, replay on demand), a perplexity-based adapter selection mechanism when task IDs are unavailable, and experimental validation across multiple task-oriented dialogue and NLP datasets showing strong performance and significant memory savings. The work demonstrates that parameter-efficient adapter storage coupled with latent-code compression supports effective knowledge transfer and resilience to catastrophic forgetting, with practical implications for deployable continual learning in NLP systems.

Abstract

Learning a set of tasks over time, also known as continual learning (CL), is one of the most challenging problems in artificial intelligence due to catastrophic forgetting. Large language models (LLMs) are often impractical to frequent re-training and continual learning , due to high cost of computational resources for training. Moreover, LLM are not suitable for continual learning as updating these models over time for acquiring new knowledge leads to overwrites existing knowledge leading to common phenomenon know as \textit{catastrophic forgetting}. In this paper, we aim to address these concerns using a novel framework , COLA that employs an autoencoder to learn capture low-dimensional embeddings of the weights associated with various tasks. Our approach facilitates the transfer of knowledge to new tasks while preventing catastrophic forgetting, all without using data replay or a substantial set of task-specific parameters. Our approach, COLA, makes the LLM efficiently learn new tasks with minimal training, insignificant performance degradation on previous tasks, and eliminates the need for retaining earlier training data. Empirical evaluation on different datasets ranging from task oriented dialouge system to intent classsfication datasets showcases that our method not only overcomes catastrophic forgetting but also achieves significant reduction in parameter usage and memory size, across multiple tasks and outperforming the existing state of the art methods across multiple datasets.

Figures (3)

• Figure 1: COLA: Our proposed framework consists of two primary components: task-specific adapter networks (TN), an optional buffer for temporarily storing the most recent $m$ tasks, and a lifelong autoencoder (AE) designed for long-term learning. Given a new task $t_k$. we first train a model with parameters $\theta_k$, We learn the optimal weights for each task and temporarily store the trained adapter in a buffer; once full, the stored adapters are encoded into the autoencoder and the buffer is cleared. Alternatively, adapters can be encoded directly without buffering. When an earlier task reoccurs (e.g., revisiting a previously learned task), we reconstruct its adapter weights by decoding the corresponding latent vector $z_i$ from the autoencoder and load them onto the task-specific adapter. Although the latent vector $z_i$ is asymptotically smaller than the original parameter set $\theta_i$, the reconstructed adapter achieves performance closely matching that of the original model.
• Figure 2: COLA: Avg Accuracy vs Baseline vs Reconstruction Score
• Figure 3: COLA: Avg Accuracy vs Baseline vs Reconstruction Score