Consistency-Driven Calibration and Matching for Few-Shot Class-Incremental Learning

TL;DR

A memory-aware prototype calibration is designed that extracts generalized semantic attributes from base classes and reintegrates them into novel classes to enhance the conceptual center consistency of features and proposes dynamic structure matching, which adaptively aligns the calibrated features to a session-specific optimal manifold space, ensuring cross-session structure consistency.

Abstract

Few-Shot Class Incremental Learning (FSCIL) is crucial for adapting to the complex open-world environments. Contemporary prospective learning-based space construction methods struggle to balance old and new knowledge, as prototype bias and rigid structures limit the expressive capacity of the embedding space. Different from these strategies, we rethink the optimization dilemma from the perspective of feature-structure dual consistency, and propose a Consistency-driven Calibration and Matching (ConCM) framework that systematically mitigates the knowledge conflict inherent in FSCIL. Specifically, inspired by hippocampal associative memory, we design a memory-aware prototype calibration that extracts generalized semantic attributes from base classes and reintegrates them into novel classes to enhance the conceptual center consistency of features. Further, to consolidate memory associations, we propose dynamic structure matching, which adaptively aligns the calibrated features to a session-specific optimal manifold space, ensuring cross-session structure consistency. This process requires no class-number priors and is theoretically guaranteed to achieve geometric optimality and maximum matching. On large-scale FSCIL benchmarks including mini-ImageNet, CIFAR100 and CUB200, ConCM achieves state-of-the-art performance, with harmonic accuracy gains of up to 3.41% in incremental sessions. Code is available at: https://github.com/wire-wqz/ConCM

Figures (7)

• Figure 1: (a) FSCIL setup and baseline. It trains the backbone through the base session and freezes it. (b) Prospective learning-based space construction method. It pre-allocate fixed embedding spaces for novel classes, subject to the a priori assumptions of the class. (c) The ConCM framework (Ours). It leverages semantic attributes to calibrate novel class features by transferring knowledge from base classes, while incorporating a dynamical mechanism of feature-structure mathcing.
• Figure 2: The proposed ConCM framework has two main modules: (a) MPC module. Generalized semantic attributes are extracted from base classes, i.e., attribute separation, followed by meta learning–based retrieval and aggregation for prototype calibration, i.e., attribute completion. (b) DSM module. Each incremental step dynamically updates embedding structure to ensure geometric optimality and maximum matching, while adaptively aligning features through loss-driven optimization.
• Figure 3: Preliminary results.We identify two issues and causes: (a) Issue 1: The accuracy on novel classes consistently declines as prototype deviation increases, caused by (b) Reason 1: Novel class prototypes deviate from the true centers, i.e., feature inconsistency. (c) Issue 2: false positive classification, caused by (d) Reason 2: feature embedding confusion, i.e., structure inconsistency.
• Figure 4: SOTA comparisons on three FSCIL benchmark datasets. Performance curve is harmonic mean accuracy. The underlined denotes the average performance improvement. The red denotes the highest performance improvement.
• Figure 5: The confusion matrix result on mini-ImageNet.