A Closer Look at Few-shot Classification

TL;DR

This work interrogates the fairness and practicality of evaluating few-shot classification methods. It shows that deeper backbones reduce apparent gains across methods and that a simple Baseline++ with a cosine-distance classifier can be highly competitive with meta-learning approaches on standard benchmarks. Crucially, in cross-domain scenarios, sophisticated meta-learning methods provide limited gains over the Baseline, underscoring the importance of adaptation to domain shifts. The authors release a unified evaluation framework to enable consistent comparisons and highlight that cross-domain few-shot learning remains a pressing, under-addressed challenge with real-world impact.

Abstract

Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples. While significant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difficult. In this paper, we present 1) a consistent comparative analysis of several representative few-shot classification algorithms, with results showing that deeper backbones significantly reduce the performance differences among methods on datasets with limited domain differences, 2) a modified baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the \miniI and the CUB datasets, and 3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classification algorithms. Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones. In a realistic cross-domain evaluation setting, we show that a baseline method with a standard fine-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.

Figures (9)

• Figure 1: Baseline and Baseline++ few-shot classification methods. Both the baseline and baseline++ method train a feature extractor $f_\theta$ and classifier $C(.|\mathbf{W}_b)$ with base class data in the training stage In the fine-tuning stage, we fix the network parameters $\theta$ in the feature extractor $f_\theta$ and train a new classifier $C(.|\mathbf{W}_n)$ with the given labeled examples in novel classes. The baseline++ method differs from the baseline model in the use of cosine distances between the input feature and the weight vector for each class that aims to reduce intra-class variations.
• Figure 2: Meta-learning few-shot classification algorithms. The meta-learning classifier $M(\cdot | \mathbf{S})$ is conditioned on the support set $\mathbf{S}$. (Top) In the meta-train stage, the support set $\mathbf{S}_b$ and the query set $\mathbf{Q}_b$ are first sampled from random $N$ classes, and then train the parameters in $M(.|\mathbf{S}_b)$ to minimize the $N$-way prediction loss $L_\mathrm{N-way}$. In the meta-testing stage, the adapted classifier $M(.|\mathbf{S}_n)$ can predict novel classes with the support set in the novel classes $\mathbf{S}_n$. (Bottom) The design of $M(\cdot | \mathbf{S})$ in different meta-learning algorithms.
• Figure 3: Few-shot classification accuracy vs. backbone depth. In the CUB dataset, gaps among different methods diminish as the backbone gets deeper. In mini-ImageNet 5-shot, some meta-learning methods are even beaten by Baseline with a deeper backbone. (Please refer to Figure \ref{['fig:backbone']} and Table \ref{['tab:backbone']} for larger figure and detailed statistics.)
• Figure 3: 5-shot accuracy under the cross-domain scenario with a ResNet-18 backbone. Baseline outperforms all other methods under this scenario.
• Figure 4: 5-shot accuracy in different scenarios with a ResNet-18 backbone. The Baseline model performs relative well with larger domain differences.