Feature-Space Generative Models for One-Shot Class-Incremental Learning

TL;DR

This work tackles one-shot class-incremental learning by introducing Gen1S, a method that learns a shared generative prior over embedding residuals from base classes and applies it to novel classes without any deployment-time training. By mapping embeddings to a residual space and conditioning on class prototypes, Gen1S uses VAEs or diffusion models to capture complex, multimodal base-class structures and generalize to unseen classes from a single example. The approach achieves state-of-the-art novel-class recognition across multiple benchmarks and backbones, with diffusion models often delivering the strongest gains as the number of novel classes increases. This demonstrates the viability of leveraging generative priors in FSCIL to enable robust one-shot adaptation on resource-constrained devices.

Abstract

Few-shot class-incremental learning (FSCIL) is a paradigm where a model, initially trained on a dataset of base classes, must adapt to an expanding problem space by recognizing novel classes with limited data. We focus on the challenging FSCIL setup where a model receives only a single sample (1-shot) for each novel class and no further training or model alterations are allowed after the base training phase. This makes generalization to novel classes particularly difficult. We propose a novel approach predicated on the hypothesis that base and novel class embeddings have structural similarity. We map the original embedding space into a residual space by subtracting the class prototype (i.e., the average class embedding) of input samples. Then, we leverage generative modeling with VAE or diffusion models to learn the multi-modal distribution of residuals over the base classes, and we use this as a valuable structural prior to improve recognition of novel classes. Our approach, Gen1S, consistently improves novel class recognition over the state of the art across multiple benchmarks and backbone architectures.

Figures (6)

• Figure 1: One-shot generalisation via diffusion model. Left: both nearest-class-mean and Gaussian classifiers collapse each T‑shaped class embeddings into a single mode, forcing a linear boundary (red solid line) that slices through the true clusters. Right: a diffusion model trained only on the residuals of the base class captures the multimodal geometry and generalizes to the novel class, yielding a nonlinear decision boundary (red dashed line) that separates the two clusters despite observing just a single sample from the novel class.
• Figure 2: Overview of our Gen1S method. We propose to train a generative model (such as a VAE or a diffusion model) on the residual embedding distribution of base classes and use it as a prior to predict novel class distributions.
• Figure 3: Distribution of L2 norm of residuals for DINOv2-s on CORe50. While samples are typically closer to correct prototypes than to incorrect ones, the considerable overlap precludes the use of distance-based approaches.
• Figure 4: PCA of DINOv2-s residuals of four CORe50 novel classes (orange) and corresponding base classes (blue) with similar structure. Distributions are not Gaussian. The structure of base classes is often similar to that of novel classes, thus yielding an informative prior.
• Figure A1: Embeddings of different classes have similar distributions of residuals. We map the original embeddings space (left plot) into a residual space by subtracting class prototypes (right plot). For visualization, we show the 2D space after PCA. We observe that base and novel class embeddings in FSCIL often exhibit similar intra-class structures. We employ a generative model to learn the shared distribution of residuals across all base classes and predict it over the single support novel class sample. Classification on query novel class samples (green cross on the left plot) is performed by minimizing the reconstruction error between per-class predicted residual and true one. In the example shown, the yellow class would be selected as output (right plot).