Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How to train your MAML

Antreas Antoniou, Harrison Edwards, Amos Storkey

TL;DR

The paper tackles the instability and inefficiency of Model Agnostic Meta-Learning (MAML) in few-shot learning. It introduces MAML++, a suite of six modifications (MSL, DA, BNRS, BNWB, LSLR, CA) to stabilize training, automate hyperparameter choices, and boost generalization. Empirical results on Omniglot and Mini-Imagenet demonstrate state-of-the-art performance across 5-way and 1/5-shot settings, with faster convergence and robustness to inner-loop updates. The findings highlight the importance of per-step parameter adaptations and per-step normalization in achieving fast, reliable few-shot learning across tasks.

Abstract

The field of few-shot learning has recently seen substantial advancements. Most of these advancements came from casting few-shot learning as a meta-learning problem. Model Agnostic Meta Learning or MAML is currently one of the best approaches for few-shot learning via meta-learning. MAML is simple, elegant and very powerful, however, it has a variety of issues, such as being very sensitive to neural network architectures, often leading to instability during training, requiring arduous hyperparameter searches to stabilize training and achieve high generalization and being very computationally expensive at both training and inference times. In this paper, we propose various modifications to MAML that not only stabilize the system, but also substantially improve the generalization performance, convergence speed and computational overhead of MAML, which we call MAML++.

How to train your MAML

TL;DR

The paper tackles the instability and inefficiency of Model Agnostic Meta-Learning (MAML) in few-shot learning. It introduces MAML++, a suite of six modifications (MSL, DA, BNRS, BNWB, LSLR, CA) to stabilize training, automate hyperparameter choices, and boost generalization. Empirical results on Omniglot and Mini-Imagenet demonstrate state-of-the-art performance across 5-way and 1/5-shot settings, with faster convergence and robustness to inner-loop updates. The findings highlight the importance of per-step parameter adaptations and per-step normalization in achieving fast, reliable few-shot learning across tasks.

Abstract

The field of few-shot learning has recently seen substantial advancements. Most of these advancements came from casting few-shot learning as a meta-learning problem. Model Agnostic Meta Learning or MAML is currently one of the best approaches for few-shot learning via meta-learning. MAML is simple, elegant and very powerful, however, it has a variety of issues, such as being very sensitive to neural network architectures, often leading to instability during training, requiring arduous hyperparameter searches to stabilize training and achieve high generalization and being very computationally expensive at both training and inference times. In this paper, we propose various modifications to MAML that not only stabilize the system, but also substantially improve the generalization performance, convergence speed and computational overhead of MAML, which we call MAML++.

Paper Structure

This paper contains 11 sections, 4 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model Agnostic Meta Learning
  4. Model Agnostic meta-learning Problems
  5. Stable, automated and improved MAML
  6. Datasets
  7. Experiments
  8. Results
  9. Conclusion
  10. Acknowledgements
  11. Additional Results

Figures (1)

  • Figure 1: Stabilizing MAML: This figure illustrates 3 seeds of the original strided MAML vs strided MAML++. One can see that 2 out of 3 seeds with the original strided MAML seem to become unstable and erratic, whereas all 3 of the strided MAML++ models seem to consistently converge very fast, to much higher generalization accuracy without any stability issues.