Adaptive Discovering and Merging for Incremental Novel Class Discovery

TL;DR

The paper tackles class-incremental Novel Class Discovery by introducing Adaptive Discovering and Merging (ADM), a two-stage framework that decouples representation learning from novel-class discovery and then merges novel knowledge into a fixed base model. It combines self-supervised contrastive learning and knowledge distillation for robust representations with Triplet Comparison and Probability Regularization to drive adaptive category assignment. For integration, ADM employs a hybrid two-branch architecture with Adaptive Feature Fusion (AFF) and Adaptive Model Merging (AMM) to achieve non-destructive learning and linear parameter fusion using BN gate signals, enabling continual learning without parameter growth. Across CIFAR-10/100 and Tiny-ImageNet, ADM-based methods outperform existing class-iNCD approaches and also benefit class-IL, demonstrating strong practical potential for scalable, open-world learning with limited overhead.

Abstract

One important desideratum of lifelong learning aims to discover novel classes from unlabelled data in a continuous manner. The central challenge is twofold: discovering and learning novel classes while mitigating the issue of catastrophic forgetting of established knowledge. To this end, we introduce a new paradigm called Adaptive Discovering and Merging (ADM) to discover novel categories adaptively in the incremental stage and integrate novel knowledge into the model without affecting the original knowledge. To discover novel classes adaptively, we decouple representation learning and novel class discovery, and use Triple Comparison (TC) and Probability Regularization (PR) to constrain the probability discrepancy and diversity for adaptive category assignment. To merge the learned novel knowledge adaptively, we propose a hybrid structure with base and novel branches named Adaptive Model Merging (AMM), which reduces the interference of the novel branch on the old classes to preserve the previous knowledge, and merges the novel branch to the base model without performance loss and parameter growth. Extensive experiments on several datasets show that ADM significantly outperforms existing class-incremental Novel Class Discovery (class-iNCD) approaches. Moreover, our AMM also benefits the class-incremental Learning (class-IL) task by alleviating the catastrophic forgetting problem.

Figures (4)

• Figure 1: Illustration of incremental novel class discovery with an elastic learner. The novel model adaptively discovers novel classes from the unlabeled data of the current task based on the previous base model. The novel learner should not only affect the knowledge of the base model but also integrate with the base model.
• Figure 2: Illustration of incremental novel class discovery with adaptive discovering and merging. (a) For incremental novel class discovery, representation learning and feature-based adaptive novel class discovery are decoupled. Knowledge distillation is applied for representation learning to mitigate catastrophic forgetting, and self-supervised contrastive learning is employed to learn new categories. For adaptive novel discovery, the base head is trained with replayed base prototypes. For the category assignment of new classes, triplet comparison enhances the probability differentiation and probability regularization promotes the diversity to avoid overfitting of categories within a single class. Finally, pseudo labels are generated by the maximum probability. (b) Sketch of adaptive model merging. Each $\mathrm{Conv}+\mathrm{BN}$ unit is decomposed into the base and novel branch, which could be merged based on the additivity of convolution.
• Figure 3: Conceptual illustrations of different merging methods. (a) Incremental Model Merging (IMM) zhu2022self decomposes each $\mathrm{CONV}+\mathrm{BN}$ operation into the base and novel parts, then fuses these two parts based on the additivity of convolution. (b) Adaptive Feature Fusion (AFF) adopts the gated output of the base branch to guide the learning process and outputs of the novel branch. (c) Adaptive Model Merging (AMM) uses the weight of base $\mathrm{BN}$ to replace dynamic gated output, so as to achieve linear merging.
• Figure 4: The max magnitude distribution of old categories on the base and novel branches.