Rethinking the Sample Relations for Few-Shot Classification

TL;DR

The paper introduces Multi-Grained Relation Contrastive Learning (MGRCL) to improve feature learning for few-shot classification by explicitly modeling three granularities of sample relations: intra-sample under transformations, intra-class homogeneous samples, and inter-class inhomogeneous samples. It implements Transformation Consistency Learning (TCL) to align label outputs across transformations and Class Contrastive Learning (CCL) to pull together homogeneous samples while pushing apart inhomogeneous ones, using a memory bank and a base classifier with a simple self-supervised branch. Across four benchmarks (miniImageNet, tieredImageNet, CIFAR-FS, CUB), MGRCL delivers competitive or state-of-the-art results and can serve as a strong pre-trained feature extractor for other FSL methods, improving performance without requiring meta-learning or generative components. The approach emphasizes a simple, effective pre-training paradigm with demonstrated benefits on both general and fine-grained datasets, while acknowledging resource considerations related to memory banks and augmentation. Overall, MGRCL provides a practical, scalable way to leverage multiple semantic relations for better few-shot generalization.

Abstract

Feature quality is paramount for classification performance, particularly in few-shot scenarios. Contrastive learning, a widely adopted technique for enhancing feature quality, leverages sample relations to extract intrinsic features that capture semantic information and has achieved remarkable success in Few-Shot Learning (FSL). Nevertheless, current few-shot contrastive learning approaches often overlook the semantic similarity discrepancies at different granularities when employing the same modeling approach for different sample relations, which limits the potential of few-shot contrastive learning. In this paper, we introduce a straightforward yet effective contrastive learning approach, Multi-Grained Relation Contrastive Learning (MGRCL), as a pre-training feature learning model to boost few-shot learning by meticulously modeling sample relations at different granularities. MGRCL categorizes sample relations into three types: intra-sample relation of the same sample under different transformations, intra-class relation of homogenous samples, and inter-class relation of inhomogeneous samples. In MGRCL, we design Transformation Consistency Learning (TCL) to ensure the rigorous semantic consistency of a sample under different transformations by aligning predictions of input pairs. Furthermore, to preserve discriminative information, we employ Class Contrastive Learning (CCL) to ensure that a sample is always closer to its homogenous samples than its inhomogeneous ones, as homogenous samples share similar semantic content while inhomogeneous samples have different semantic content. Our method is assessed across four popular FSL benchmarks, showing that such a simple pre-training feature learning method surpasses a majority of leading FSL methods. Moreover, our method can be incorporated into other FSL methods as the pre-trained model and help them obtain significant performance gains.

Figures (6)

• Figure 1: In this figure, shapes and colors represent different samples and classes respectively. Different transformations of the same sample are represented by the same color and shape. The sample relations contain three types: intra-sample relation of the same sample under different transformations, intra-class relation of homogenous samples, and inter-class relation of inhomogeneous samples. Our approach enforces different transformations to be consistent in semantic content, homogenous samples to be similar, and inhomogeneous samples to be different. Unsupervised contrastive learning only constrains the intra-sample relation, and supervised contrastive learning treats the intra-sample relation as equal to the intra-class relation.
• Figure 2: Network architecture of our model, which contains a CNN backbone $\mathcal{F}_{\theta}$ and a classifier $\mathcal{F}_{\varphi}$. Our proposed Transformation Consistency Learning and Class Contrastive Learning constrain different sample relations at the label level and feature level, respectively. In this figure, $v$ and $v^{(m)}$ represent the feature embeddings of the original image and its $m$-th transformed version. $\bigoplus$ is a concatenation operator for the predicted output $\widehat{y}$ of the original image and the predicted outputs $\{\widehat{y}^{(1)}, ..., \widehat{y}^{(M)}$} of $M$ transformations. $\widehat{y}_{all}$ and $y_{all}$ are the predicted outputs and the ground truths of the original image and its transformations. And memory bank is used to store the feature embeddings.
• Figure 3: Effects of hyper-parameters $\alpha$ and $\beta$ on miniImageNet. (a) $\alpha$. (b) $\beta$.
• Figure 4: Effects of hyper-parameters $\tau_1$ and $\tau_2$ on miniImageNet. (a) $\tau_1$. (b) $\tau_2$.
• Figure 5: Comparison to the base learner with Unsupervised Contrastive Learning or Supervised Contrastive Learning in 5-way 1-shot tasks on miniImageNet and CUB. (a) miniImageNet. (b) CUB.