Brain-inspired analogical mixture prototypes for few-shot class-incremental learning

TL;DR

Few-shot class-incremental learning (FSCIL) remains challenging due to catastrophic forgetting and overfitting when incrementally learning from limited data. The authors propose Brain-inspired Analogical Mixture Prototypes (BAMP), a three-component framework built on a pre-trained Vision Transformer (ViT) that combines mixed prototypical feature learning, statistical analogy to calibrate new-class representations, and soft voting to fuse signals from multiple sources. Prototypes are modeled as mixtures of von Mises-Fisher distributions on the hypersphere; incremental testing calibrates means and covariances via analogy to base classes and uses Mahalanobis distance for scoring, while soft voting blends these scores with an off-the-shelf FSCIL classifier. Across six FSCIL benchmarks under both big-start and small-start settings, BAMP achieves state-of-the-art performance, with ablations confirming the contribution of each component. The approach offers a brain-inspired mechanism to alleviate forgetting and overfitting, providing robust generalization across diverse domains and tasks.

Abstract

Few-shot class-incremental learning (FSCIL) poses significant challenges for artificial neural networks due to the need to efficiently learn from limited data while retaining knowledge of previously learned tasks. Inspired by the brain's mechanisms for categorization and analogical learning, we propose a novel approach called Brain-inspired Analogical Mixture Prototypes (BAMP). BAMP has three components: mixed prototypical feature learning, statistical analogy, and soft voting. Starting from a pre-trained Vision Transformer (ViT), mixed prototypical feature learning represents each class using a mixture of prototypes and fine-tunes these representations during the base session. The statistical analogy calibrates the mean and covariance matrix of prototypes for new classes according to similarity to the base classes, and computes classification score with Mahalanobis distance. Soft voting combines both merits of statistical analogy and an off-shelf FSCIL method. Our experiments on benchmark datasets demonstrate that BAMP outperforms state-of-the-art on both traditional big start FSCIL setting and challenging small start FSCIL setting. The study suggests that brain-inspired analogical mixture prototypes can alleviate catastrophic forgetting and over-fitting problems in FSCIL.

Figures (4)

• Figure 1: Schematic of the brain-inspired analogical mixture prototypes. A The biological mechanisms of categorization and analogical learning. B The brain-inspired analogical mixture prototypes for FSCIL.
• Figure 2: Average of $A_{last}$ and $A_{inc}$ over 6 datasets. "Final" refers to the classification accuracy (ACC) for all seen classes in the last session. "Average" refers to the mean ACC across all sessions. A is for the big start setting, while B is for the small start setting.
• Figure 3: Accuracy after each incremental session for big start setting on six datasets.
• Figure 4: Accuracy after each incremental session for small start setting on six datasets.