Evolutionary Machine Learning meets Self-Supervised Learning: a comprehensive survey

TL;DR

This survey formalizes Evolutionary Self-Supervised Learning (E-SSL) as the intersection of Evolutionary Machine Learning and Self-Supervised Learning, and introduces a taxonomy that separates EC for SSL and SSL for EC. It catalogs 72 papers, highlighting how evolution can automate pretext-task design, data selection, topology, and learning in SSL, as well as how SSL techniques can enhance evolutionary processes via learned representations, operators, and surrogate fitness. The authors identify key challenges such as pretext-task automation, evaluation under label scarcity, and efficient fitness evaluation, and they propose directions to broaden meta-architectures and measurement methodologies. By articulating these themes, the paper aims to catalyse further research toward robust, label-efficient, and scalable E-SSL methods with practical impact across domains.

Abstract

The number of studies that combine Evolutionary Machine Learning and self-supervised learning has been growing steadily in recent years. Evolutionary Machine Learning has been shown to help automate the design of machine learning algorithms and to lead to more reliable solutions. Self-supervised learning, on the other hand, has produced good results in learning useful features when labelled data is limited. This suggests that the combination of these two areas can help both in shaping evolutionary processes and in automating the design of deep neural networks, while also reducing the need for labelled data. Still, there are no detailed reviews that explain how Evolutionary Machine Learning and self-supervised learning can be used together. To help with this, we provide an overview of studies that bring these areas together. Based on this growing interest and the range of existing works, we suggest a new sub-area of research, which we call Evolutionary Self-Supervised Learning and introduce a taxonomy for it. Finally, we point out some of the main challenges and suggest directions for future research to help Evolutionary Self-Supervised Learning grow and mature as a field.

Figures (9)

• Figure 1: Number of surveyed publications related with breakdown by year
• Figure 2: Overview of the process for an image classification problem.
• Figure 3: Relation of field when compared to , and
• Figure 4: GenNAS for convolutional neural network architectures li2021generic. Each stage contains a single convolutional layer $M$ whose feature maps are optimised to be as approximated as possible to synthetic signals.
• Figure 5: Example of an asymmetric design allowed by Assunção et al. assuncao2018automatic. The layer with the lowest dimensionality is the one that defines the representations.