Meta-Learning: A Survey

TL;DR

Meta-learning surveys how to learn from prior experience across tasks by collecting meta-data (configurations, evaluations, model parameters, and meta-features) and using it to accelerate new-task learning. It organizes approaches by the type of meta-data used, detailing evaluation-driven methods, task-property features, and prior-model transfer, including both traditional Bayesian and modern neural meta-learning techniques. Key contributions include strategies for task similarity, surrogate models, learning curves transfer, pipeline synthesis, and meta-models for ranking and performance prediction, as well as foundations for transfer and few-shot learning in neural networks. The chapter highlights AutoML implications, showing how data-driven meta-experience can dramatically speed up hyperparameter tuning, model selection, and pipeline construction across varied domains.

Abstract

Meta-learning, or learning to learn, is the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks, and then learning from this experience, or meta-data, to learn new tasks much faster than otherwise possible. Not only does this dramatically speed up and improve the design of machine learning pipelines or neural architectures, it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way. In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.