BECLR: Batch Enhanced Contrastive Few-Shot Learning

TL;DR

BECLR addresses unsupervised few-shot learning by coupling two novel components: Dynamic Clustered Memory (DyCE) for class-aware pretraining and Optimal Transport-based Distribution Alignment (OpTA) for bias-aware inference. DyCE constructs a dynamically updated memory with prototypes to create meaningful positives, yielding a highly separable latent space for contrastive learning. OpTA aligns the distributions of support and query sets during inference via a non-parametric optimal transport step, mitigating sample bias in low-shot regimes. Together, BECLR achieves state-of-the-art performance across multiple unsupervised FSL benchmarks and demonstrates strong cross-domain generalization, with reproducible results and publicly available code.

Abstract

Learning quickly from very few labeled samples is a fundamental attribute that separates machines and humans in the era of deep representation learning. Unsupervised few-shot learning (U-FSL) aspires to bridge this gap by discarding the reliance on annotations at training time. Intrigued by the success of contrastive learning approaches in the realm of U-FSL, we structurally approach their shortcomings in both pretraining and downstream inference stages. We propose a novel Dynamic Clustered mEmory (DyCE) module to promote a highly separable latent representation space for enhancing positive sampling at the pretraining phase and infusing implicit class-level insights into unsupervised contrastive learning. We then tackle the, somehow overlooked yet critical, issue of sample bias at the few-shot inference stage. We propose an iterative Optimal Transport-based distribution Alignment (OpTA) strategy and demonstrate that it efficiently addresses the problem, especially in low-shot scenarios where FSL approaches suffer the most from sample bias. We later on discuss that DyCE and OpTA are two intertwined pieces of a novel end-to-end approach (we coin as BECLR), constructively magnifying each other's impact. We then present a suite of extensive quantitative and qualitative experimentation to corroborate that BECLR sets a new state-of-the-art across ALL existing U-FSL benchmarks (to the best of our knowledge), and significantly outperforms the best of the current baselines (codebase available at: https://github.com/stypoumic/BECLR).

Figures (10)

• Figure 1: miniImageNet ($5$-way, $1$-shot, left) and ($5$-way, $5$-shot, right) accuracy in the U-FSL landscape. BECLR sets a new state-of-the-art in all settings by a significant margin.
• Figure 2: Overview of the proposed pretraining framework of BECLR. Two augmented views of the batch images $\bm{X}^{\{\alpha,\beta\}}$ are both passed through a student-teacher network followed by the DyCE memory module. DyCE enhances the original batch with meaningful positives and dynamically updates the memory partitions.
• Figure 3: Overview of the proposed dynamic clustered memory (DyCE) and its two informational paths.
• Figure 4: Overview of the inference strategy of BECLR. Given a test episode, the support ($\mathcal{S}$) and query ($\mathcal{Q}$) sets are passed to the pretrained feature extractor ($f_\theta$). OpTA aligns support prototypes and query features.
• Figure 5: BECLR outperforms all baselines, in terms of U-FSL performance on miniImageNet, even without OpTA.