Curriculum Learning: A Survey
Petru Soviany, Radu Tudor Ionescu, Paolo Rota, Nicu Sebe
TL;DR
This survey unifies curriculum learning (CL) under a generic optimization framework, presents a seven-category taxonomy, and validates its structure with a hierarchical clustering of nearly 200 works. It demonstrates that easy-to-hard curricula can accelerate convergence and improve accuracy across computer vision, natural language processing, speech, medical imaging, and reinforcement learning, while also highlighting limitations such as difficulty estimation and data diversity concerns. The authors contribute a formal CL formulation, taxonomy, and data-driven clustering insights, together with practical directions for future work. They also emphasize the need to explore model- and performance-level curricula and extend CL into unsupervised and self-supervised learning paradigms.
Abstract
Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs. Curriculum learning strategies have been successfully employed in all areas of machine learning, in a wide range of tasks. However, the necessity of finding a way to rank the samples from easy to hard, as well as the right pacing function for introducing more difficult data can limit the usage of the curriculum approaches. In this survey, we show how these limits have been tackled in the literature, and we present different curriculum learning instantiations for various tasks in machine learning. We construct a multi-perspective taxonomy of curriculum learning approaches by hand, considering various classification criteria. We further build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm, linking the discovered clusters with our taxonomy. At the end, we provide some interesting directions for future work.