Parameter-Efficient Augment Plugin for Class-Incremental Learning

TL;DR

This paper tackles catastrophic forgetting in class-incremental learning by proposing DLC, a parameter-efficient plugin framework that injects task-specific residuals via ConvLoRA adapters into a frozen base model trained with replay and distillation. A dynamic weighting unit gates the contributions from multiple task plugins during inference, mitigating interference from non-target residuals. The two-phase training scheme keeps backbone updates decoupled from plugin learning, enabling seamless integration with non-expansion CIL baselines and delivering notable accuracy gains with modest parameter overhead (e.g., 8% on ImageNet-100 with only ~4% extra parameters). DLC also demonstrates compatibility with additional plug-and-play enhancements and under fixed memory budgets, offering a scalable, practical path toward high-performance continual learning without backbone expansion.

Abstract

Existing class-incremental learning (CIL) approaches based on replay or knowledge distillation are often constrained by forgetting or the stability-plasticity dilemma. Some expansion-based approaches could achieve higher accuracy. However, they always require significant parameter increases. In this paper, we propose a plugin extension paradigm termed the Deployment of extra LoRA Components (DLC) for non-pre-trained CIL scenarios.We treat the feature extractor trained through replay or distillation as a base model with rich knowledge. For each task, we use Low-Rank Adaptation (LoRA) to inject task-specific residuals into the base model's deep layers. During inference, representations with task-specific residuals are aggregated to produce classification predictions. To mitigate interference from non-target LoRA plugins, we introduce a lightweight weighting unit. This unit learns to assign importance scores to different LoRA-tuned representations. Like downloadable contents in software, our method serves as a plug-and-play enhancement that efficiently extends the base methods. Remarkably, on the large-scale ImageNet-100, with merely 4 % of the parameters of a standard ResNet-18, our DLC model achieves a significant 8 % improvement in accuracy, demonstrating exceptional efficiency. Moreover, it could surpass state-of-the-art methods under the fixed memory budget.

Figures (6)

• Figure 1: Parameter-accuracy comparison of different CIL methods on ImageNet-100 with ResNet-18. DLC achieves the best balance between parameter efficiency and accuracy compared to prior SOTA methods. iCaRL-DLC, WA-DLC, and BiC-DLC mean integrating DLC into the corresponding methods, respectively.
• Figure 2: The proposed DLC framework. Left: Training. When a new task $t$ arrives, a dedicated plugin set $L_t$ is created. Then sequentially train the feature extractor and $L_t$. Right: Test. The feature extractor sequentially loads all task plugins $L_t$ to produce enhanced representations, which are concatenated, weighted by a gating unit, and classified.
• Figure 3: Confusion matrix heatmaps of iCaRL with and w/o DLC on CIFAR-100 B10 Inc10.
• Figure 4: Comparison of $A_T$ with/without DLC on CIFAR-100 when enhanced with AutoAugment (AA) and $\mathcal{L}_{\text{inter-intra}}$.
• Figure 5: Comparison of $A_T$ with and without the weighting unit for DLC-enhanced methods on CIFAR-100.