Specifying What You Know or Not for Multi-Label Class-Incremental Learning

TL;DR

The paper tackles multi-label class-incremental learning (MLCIL), where incomplete labels across $T$ incremental sessions create conflicts among preserving past knowledge, learning current classes, and preparing for future ones. It introduces HCP, a framework that specifies what is known or not, using (i) dynamic feature purification with class embeddings to form fine-grained, aliasing-free known features, (ii) recall enhancement with distribution priors to stabilize old knowledge, and (iii) probing unknown knowledge by synthesizing future-like features to enrich the feature space. The approach yields state-of-the-art results on MS-COCO and PASCAL VOC, notably surpassing prior methods by up to $3.3 ext{ percentage points}$ in Avg Acc on MS-COCO under the $B0 ext{-}C10$ setting without replay buffers, and demonstrates robust improvements across protocols with or without buffers. The work provides a new lens for MLCIL by explicitly delineating known versus unknown knowledge, improving inter-class discriminability and forward compatibility for long-term incremental learning.

Abstract

Existing class incremental learning is mainly designed for single-label classification task, which is ill-equipped for multi-label scenarios due to the inherent contradiction of learning objectives for samples with incomplete labels. We argue that the main challenge to overcome this contradiction in multi-label class-incremental learning (MLCIL) lies in the model's inability to clearly distinguish between known and unknown knowledge. This ambiguity hinders the model's ability to retain historical knowledge, master current classes, and prepare for future learning simultaneously. In this paper, we target at specifying what is known or not to accommodate Historical, Current, and Prospective knowledge for MLCIL and propose a novel framework termed as HCP. Specifically, (i) we clarify the known classes by dynamic feature purification and recall enhancement with distribution prior, enhancing the precision and retention of known information. (ii) We design prospective knowledge mining to probe the unknown, preparing the model for future learning. Extensive experiments validate that our method effectively alleviates catastrophic forgetting in MLCIL, surpassing the previous state-of-the-art by 3.3% on average accuracy for MS-COCO B0-C10 setting without replay buffers.

Figures (7)

• Figure 1: The contradiction of learning objectives in MLCIL arises from the model's inability to distinguish known and unknown knowledge. Current model fails to effectively recall prior known knowledge due to (a) the absence of historical labels, while (b) unknown classes' attention is inadvertently overlapped with known classes, and known classes are also entangled, resulting in (c) feature aliasing and contradictory learning objectives. By specifying what is known or not, (d) fine-grained class-aware features are focused, leading to (e) enhanced inter-class discriminability, alleviating the contradiction.
• Figure 2: Framework of HCP, which leverages Clarifying Known and Probing Unknown to accommodate historical, current, and prospective knowledge. For clarifying known knowledge, we design dynamic Feature Purification to capture fine-grained class-aware features $O_{s}$ to avoid feature aliasing across sessions, and Recall Enhancement with distribution prior to effectively retain historical known knowledge. For probing unknown knowledge, we interpolate known features as prospective class to help enrich the feature set, enhancing the discriminability of known features and facilitate future learning.
• Figure 3: Illustration of feature purification. Each session appends new class embeddings $S_{t}$ for new class features $O_{St}$.
• Figure 4: Confidence forgetting varies greatly among classes, making it difficult to effectively recall known knowledge by a unified and static pseudo-label threshold.
• Figure 5: Performance curves (mAP%) on MS-COCO and PASCAL VOC datasets under different protocols.