Cross-Domain Continual Learning via CLAMP

TL;DR

CLAMP addresses the challenging problem of cross-domain continual learning by jointly mitigating catastrophic forgetting and domain shift. It introduces a dual-assessor meta-learning framework that soft-weights three losses and enables class-aware adversarial domain adaptation to align source and target representations while learning from a labelled source domain to support an unlabelled target domain. The method demonstrates substantial improvements over a broad set of baselines across multiple benchmarks, with thorough ablations, memory analyses, and theoretical generalization bounds. This approach offers a practical pathway for deploying a single model across evolving, partially labelled environments with reduced labeling costs and robust performance. The work also provides public code and extensive analyses, highlighting CLAMP’s potential for real-world continual learning under domain drift, with clear avenues to reduce complexity and extend to more realistic privacy-preserving settings.

Abstract

Artificial neural networks, celebrated for their human-like cognitive learning abilities, often encounter the well-known catastrophic forgetting (CF) problem, where the neural networks lose the proficiency in previously acquired knowledge. Despite numerous efforts to mitigate CF, it remains the significant challenge particularly in complex changing environments. This challenge is even more pronounced in cross-domain adaptation following the continual learning (CL) setting, which is a more challenging and realistic scenario that is under-explored. To this end, this article proposes a cross-domain CL approach making possible to deploy a single model in such environments without additional labelling costs. Our approach, namely continual learning approach for many processes (CLAMP), integrates a class-aware adversarial domain adaptation strategy to align a source domain and a target domain. An assessor-guided learning process is put forward to navigate the learning process of a base model assigning a set of weights to every sample controlling the influence of every sample and the interactions of each loss function in such a way to balance the stability and plasticity dilemma thus preventing the CF problem. The first assessor focuses on the negative transfer problem rejecting irrelevant samples of the source domain while the second assessor prevents noisy pseudo labels of the target domain. Both assessors are trained in the meta-learning approach using random transformation techniques and similar samples of the source domain. Theoretical analysis and extensive numerical validations demonstrate that CLAMP significantly outperforms established baseline algorithms across all experiments by at least $10\%$ margin.

Figures (7)

• Figure 1: Cross-Domain Continual Learning Problems
• Figure 2: CLAMP architecture includes the main module (encoder $f_{\theta}$, classifier $g_{\phi}$ and domain classifier ${\xi}_{\psi}$) and two assessors ($\zeta_S$ and $\zeta_T$). In cross-domain continual learning scenario, the adversarial learning process of the base learner, encoder and classifier is adjusted by two assessors trained in the meta-learning method. Domain classifier of main network aims to solve unsupervised domain adaptation.
• Figure 3: Average query accuracy comparison across different baselines for four benchmarks: digit recognition, Office-31, Office-Home and VisDA
• Figure 4: t-SNE plots of the USPS (in light colors) $\rightarrow$ MNIST (in dark colors) case
• Figure 5: CLAMP loss trajectories in digits recognition: (a) MNIST$\rightarrow$USPS, (b) USPS$\rightarrow$MNIST. It illustrates the trajectories of training loss and validation loss over the entire training process in the cross-domain continual learning scenario.