Confidence Self-Calibration for Multi-Label Class-Incremental Learning

TL;DR

This paper introduces a class-incremental graph convolutional network that bridges the isolated label spaces by constructing learnable, dynamically extended label relationship graph, and presents a max-entropy regularization for each multi-label increment, facilitating confidence self-calibration through the penalization of over-confident output distributions.

Abstract

The partial label challenge in Multi-Label Class-Incremental Learning (MLCIL) arises when only the new classes are labeled during training, while past and future labels remain unavailable. This issue leads to a proliferation of false-positive errors due to erroneously high confidence multi-label predictions, exacerbating catastrophic forgetting within the disjoint label space. In this paper, we aim to refine multi-label confidence calibration in MLCIL and propose a Confidence Self-Calibration (CSC) approach. Firstly, for label relationship calibration, we introduce a class-incremental graph convolutional network that bridges the isolated label spaces by constructing learnable, dynamically extended label relationship graph. Then, for confidence calibration, we present a max-entropy regularization for each multi-label increment, facilitating confidence self-calibration through the penalization of over-confident output distributions. Our approach attains new state-of-the-art results in MLCIL tasks on both MS-COCO and PASCAL VOC datasets, with the calibration of label confidences confirmed through our methodology.

Figures (7)

• Figure 1: (a) A simple depiction of multi-label class-incremental learning. There are a total of three labels, including "cat", "dog" and "person", which are sequentially learned and tested across three tasks. (b) After training on task 3, given a test image only with "person" processing by $f_{\theta^3}$. Left: attention maps for the labels "dog" and "person" using the SOTA methods PODNet douillard2020podnet, AGCN 10221710 and ours. Right: The confident scores for the three methods. The false-positive (F-P) error occurs on the "dog" using PODNet and AGCN. The bottom shows the effectiveness of our method.
• Figure 2: Overview of confidence self-calibration framework. Given an image, we first employ a CNN backbone to extract the class-agnostic feature maps F. Then, inspired by CAM zhou2016learning, we use the class activation maps as masks to decouple F into class-aware label representations $\textbf{V}_0$. Our model constructs general and specific graphs among $\textbf{V}_0$ to generate $\textbf{V}_2$, which aggregates rich cross-task label relationships. Finally, we combine graph-based representations $\hat{y}^t_\text{gcn}$ with class-related outputs $\hat{y}^t_\text{cls}$ for classification.
• Figure 3: (a) The quantitative results without or with max-entropy based on CSC framework after training on the final task. CR-CP represents per-class recall minus precision and OR-OP represents overall recall minus precision. (b) The corresponding multi-label confidence statistics on the entire test set, as the information entropy $H$ increases from 1.61 (w/o H) to 5.36 (w/ H) in {B4-C2} of VOC 2007.
• Figure 4: Comparison results on MS-COCO and VOC datasets in challenging scenarios. There are more incremental tasks in these scenarios.
• Figure 5: Correlation matrix study. "Z": fixed statistical CM, "G": general CM and "S": specific CM.