Any-Way Meta Learning

TL;DR

This work identifies a fundamental rigidity in traditional meta-learning: fixed task cardinality during training limits adaptability to unseen cardinalities. It introduces an any-way meta-learning framework that leverages label equivalence to operate over a larger output space with assignment-based losses, achieving competitive or superior performance and faster convergence across diverse benchmarks. To address semantic gaps inherent in purely label-based supervision, the authors inject semantic information via a semantic classifier and augment with Mixup, enabling cross-domain transfer and improved robustness for both GBML and MBML approaches such as MAML and ProtoNet. The results demonstrate that any-way meta-learning, especially when combined with ensemble-like label assignment and semantic augmentation, offers a practical, scalable direction for domain-general rapid adaptation in few-shot settings.

Abstract

Although meta-learning seems promising performance in the realm of rapid adaptability, it is constrained by fixed cardinality. When faced with tasks of varying cardinalities that were unseen during training, the model lacks its ability. In this paper, we address and resolve this challenge by harnessing `label equivalence' emerged from stochastic numeric label assignments during episodic task sampling. Questioning what defines ``true" meta-learning, we introduce the ``any-way" learning paradigm, an innovative model training approach that liberates model from fixed cardinality constraints. Surprisingly, this model not only matches but often outperforms traditional fixed-way models in terms of performance, convergence speed, and stability. This disrupts established notions about domain generalization. Furthermore, we argue that the inherent label equivalence naturally lacks semantic information. To bridge this semantic information gap arising from label equivalence, we further propose a mechanism for infusing semantic class information into the model. This would enhance the model's comprehension and functionality. Experiments conducted on renowned architectures like MAML and ProtoNet affirm the effectiveness of our method.

Figures (3)

• Figure 1: Task sampling procedure in our any-way meta-learning. Meta-learning employs episodic learning using a mother dataset with numerous classes ($M$) where each label represents each semantic class (e.g. 'turtle', 'octopus', and 'deer'). Each episode samples $N < M$ classes from this dataset and assigns a random numeric class ($\in [N]$) to each selected class. The flexibility of our any-way setting allows varying $N$ across tasks, distinguishing it from the conventional fixed sampling procedure.
• Figure 2: Overall structure of our proposed algorithm. The upper section outlines the implementation of the any-way meta-learning, while the lower section highlights the integration of semantic class information. Note that the encoder $f$ is not updated with the semantic classifier $g_s$ during the inner loop (dotted green line). For mixup input, a separate numeric label is assigned in $g_a$, while traditional mixup training is done in $g_s$.
• Figure 3: A graph comparing validation accuracy upon training epochs. The numbers in parentheses indicate the number of shots. The validation accuracy is measured in the 5-way case. mini refers MiniImageNet.