Towards Task Sampler Learning for Meta-Learning

TL;DR

This work challenges the notion that simply increasing task diversity universally improves meta-learning generalization. It introduces Adaptive Sampler (ASr), a plug-and-play module that learns per-episode task weights from three quality measures—task diversity, task entropy, and task difficulty—to produce an optimal training distribution. The authors provide empirical and theoretical evidence that no single sampler fits all settings, and demonstrate ASr’s broad effectiveness across standard, cross-domain, multi-domain, and FSCIL scenarios with diverse meta-learning models. The results show ASr often matches or surpasses state-of-the-art samplers with minimal overhead, and ablations confirm all three measurements contribute meaningfully. Together, these contributions offer a principled, practical approach to task sampling that enhances generalization while reducing risk of under-/over-fitting.

Abstract

Meta-learning aims to learn general knowledge with diverse training tasks conducted from limited data, and then transfer it to new tasks. It is commonly believed that increasing task diversity will enhance the generalization ability of meta-learning models. However, this paper challenges this view through empirical and theoretical analysis. We obtain three conclusions: (i) there is no universal task sampling strategy that can guarantee the optimal performance of meta-learning models; (ii) over-constraining task diversity may incur the risk of under-fitting or over-fitting during training; and (iii) the generalization performance of meta-learning models are affected by task diversity, task entropy, and task difficulty. Based on this insight, we design a novel task sampler, called Adaptive Sampler (ASr). ASr is a plug-and-play module that can be integrated into any meta-learning framework. It dynamically adjusts task weights according to task diversity, task entropy, and task difficulty, thereby obtaining the optimal probability distribution for meta-training tasks. Finally, we conduct experiments on a series of benchmark datasets across various scenarios, and the results demonstrate that ASr has clear advantages.

Figures (12)

• Figure 1: Task Samplers. These twelve samplers are divided into five categories, i.e., (i) standard sampler: Uniform sampler; (ii) low-diversity task samplers: NDT, NDE, NDTE, and SEU; (iii) high-diversity task samplers: OHTM, OWHTM, s-DPP, and d-DPP; (iv) previously proposed adaptive task samplers: GCP and DATS; and (v) our proposed Adaptive Sampler (ASr). More details are shown in Section \ref{['sec:3.3']}.
• Figure 2: The training process of MAML on various meta-learning settings, including standard few-shot classification (Omniglot), cross-domain few-shot classification (miniImagenet $\to$ CUB), multi-domain few-shot classification (Meta-Dataset), and few-shot regression (Sinusoid).
• Figure 3: Average accuracy of meta-learning models with 10 task samplers on miniImagenet. The $\bigtriangledown$ ($\bigtriangleup$) indicate that the accuracy of using the current sampler is lower (higher) than that of using Uniform Sampler.
• Figure 4: Ablation study of ASr on 4 benchmarks. The meta-learning model in this experiment is MAML.
• Figure 5: The results of meta-learning models trained on tasks with different task entropy levels.

Theorems & Definitions (7)

• corollary thmcountercorollary
• theorem 1
• theorem 2
• theorem 3
• theorem 4
• theorem 5
• theorem 6