DualCP: Rehearsal-Free Domain-Incremental Learning via Dual-Level Concept Prototype

TL;DR

DualCP tackles rehearsal-free domain-incremental learning by representing cross-domain images of the same class as a single concept and leveraging dual-level prototypes to improve separability. The framework builds coarse-grained prototypes across groups and fine-grained prototypes within groups from text features, then aligns image features via a coarse-to-fine calibrator trained with a Dual Dot-Regression loss. A theoretical result shows dual-level prototypes yield larger inter-class angles than vanilla prototypes, supporting enhanced discrimination. Extensive experiments on DomainNet, CDDB, and CORe50 demonstrate consistent gains over state-of-the-art rehearsal-free methods and competitive performance with rehearsal-based approaches, underscoring the approach’s practicality for privacy-preserving, scalable domain-incremental learning.

Abstract

Domain-Incremental Learning (DIL) enables vision models to adapt to changing conditions in real-world environments while maintaining the knowledge acquired from previous domains. Given privacy concerns and training time, Rehearsal-Free DIL (RFDIL) is more practical. Inspired by the incremental cognitive process of the human brain, we design Dual-level Concept Prototypes (DualCP) for each class to address the conflict between learning new knowledge and retaining old knowledge in RFDIL. To construct DualCP, we propose a Concept Prototype Generator (CPG) that generates both coarse-grained and fine-grained prototypes for each class. Additionally, we introduce a Coarse-to-Fine calibrator (C2F) to align image features with DualCP. Finally, we propose a Dual Dot-Regression (DDR) loss function to optimize our C2F module. Extensive experiments on the DomainNet, CDDB, and CORe50 datasets demonstrate the effectiveness of our method.

Figures (5)

• Figure 1: (a) Illustration of most existing methods, where new domain features and old domain features compete within the feature space. (b) Illustration of our method based on dual-level concept prototypes. Different colors (red, green, blue, and purple) represent different classes, while different shapes ($\circ$, $\square$, and $\diamond$) represent different domains. Best viewed in color.
• Figure 2: The framework of the proposed DualCP. DualCP comprises three main components: (a) a feature extractor to get the image features, (b) the CPG module to construct the dual-level concept prototype based on the text features of the class names, and (c) the C2F module to align the image features with the corresponding prototypes. We selected six common classes from the DomainNet dataset, i.e., cat, flower, dog, boat, bike, and bus, to further illustrate our method. Similar classes were grouped, such as cats and dogs. We constructed coarse-grained prototypes between groups and fine-grained prototypes within groups. This coarse-to-fine classification approach helps the model better distinguish similar categories. Best viewed in color.
• Figure 3: Ablation study of the C2F module on DomainNet dataset.$A_{T}$, $F_{T}$ denotes the average accuracy and the forgetting degree, respectively. The hidden dimensions are set as multiples of the image feature dimensions of 768 in the ViT-B/16 backbone. "0.5x, 1x, 2x, 4x" correspond to hidden dimensions of 384, 768, 1536, and 3072, respectively.
• Figure 4: t-SNE visualization of feature space for common classes in DomainNet, with flower represented in green, cat and dog in (c), and boat, bicycle, and bus in (d). The pentagrams represent the average image features of a group.
• Figure 5: Ablation study of the hyperparameter $p$ on the DomainNet dataset. When the hyperparameter $p$ is set to 1, it implies that each group contains only one class. In this case, our DualCP method degenerates into VanillaCP.