Aggregated f-average Neural Network applied to Few-Shot Class Incremental Learning

TL;DR

The paper tackles few-shot class incremental learning (FSCIL) by eschewing heavy meta-learning in favor of a principled ensemble fusion. It introduces Aggregated $f$-averages (AFA), a shallow neural-network that aggregates multiple $f$-averages with learnable weights to fuse predictions from session-specific weak classifiers, while using padding and inlierness to handle evolving class spaces. The approach demonstrates competitive or superior performance across mini-ImageNet, tiered-ImageNet, CUB-200, and FGVC-Aircraft, with strong interpretability and memory efficiency due to a compact parameterization and prototype-rehearsal training. The work emphasizes robust evaluation through base/new class metrics and F1 scores, and suggests extending AFA to regression tasks as future work.

Abstract

Ensemble learning leverages multiple models (i.e., weak learners) on a common machine learning task to enhance prediction performance. Basic ensembling approaches average the weak learners outputs, while more sophisticated ones stack a machine learning model in between the weak learners outputs and the final prediction. This work fuses both aforementioned frameworks. We introduce an aggregated f-average (AFA) shallow neural network which models and combines different types of averages to perform an optimal aggregation of the weak learners predictions. We emphasise its interpretable architecture and simple training strategy, and illustrate its good performance on the problem of few-shot class incremental learning.

Figures (7)

• Figure 1: This work addresses few-shot class incremental learning by training an ensemble of weak classifiers, each specialised in their own set of classes from their given session. For each session $k$, the weak classifier is composed of a feature extractor $f_\theta$ (trained only in base session and frozen in subsequent sessions) and a classification head $H_{W_k}$ (fitted on data from their own new session $k$). A dynamic padding compensates the discrepancy between the sizes of prediction vectors at each session. The resulting set of weak classifiers is ensembled by our proposed Aggregated $f$-averages (AFA) model. AFA neural network is designed to model and combine different types of averages, in an automatic supervised fashion, in order to perform an optimal output fusion.
• Figure 2: Structure of the $f$-average network, i.e. a neural network that performs an $f$-average for ensembling
• Figure 3: Structure of the proposed aggregated $f$-average (AFA) neural network. It aggregates $J$$f$-averages for ensembling, with $A \in [0, +\infty)^{N \times NJ}$. The activation function $g: \mathbb{R}^{N} \rightarrow \mathbb{R}^{N}$ is selected according to the task (e.g, softmax for classification, linear for regression).
• Figure 4: AFA ensembling for FSCIL. The CNN feature extractor $f_\theta$ is trained in base session and frozen for following sessions. Its classification head $H_{W_1}$ is a standard classification layer. For subsequent sessions (for $k \in \{2,\ldots,K\}$) SimpleShot classifications heads $H_{W_k}$ are used, based on nearest-neighbour classifiers, and fine-tuned using $D_k^\text{train}$. A padding $p$ is added to compensate the size of outputs $x$ from classification heads.
• Figure 5: F1 comparison with the state of the art on FGVC-Aircraft and tiered-ImageNet