Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities

TL;DR

This survey addresses the problem of slow convergence and suboptimal generalization in traditional evolutionary algorithms by surveying reinforcement learning–assisted evolutionary algorithms (RL-EA). It systematically categorizes RL-EA along integration methods (direct vs indirect), RL-assisted strategies (solution generation, learnable objectives, operator/algorithm/sub-population selection, parameter adaptation, etc.), and RL attribute settings (state, reward, action). The paper covers a wide range of applications in continuous and combinatorial optimization, including production scheduling and vehicle routing, and discusses open datasets and practical performance improvements. It also identifies challenges and outlines future directions, such as multi-agent RL, DRL considerations, theoretical analyses, and benchmark development, to advance RL-EA research and application.

Abstract

Evolutionary algorithms (EA), a class of stochastic search methods based on the principles of natural evolution, have received widespread acclaim for their exceptional performance in various real-world optimization problems. While researchers worldwide have proposed a wide variety of EAs, certain limitations remain, such as slow convergence speed and poor generalization capabilities. Consequently, numerous scholars actively explore improvements to algorithmic structures, operators, search patterns, etc., to enhance their optimization performance. Reinforcement learning (RL) integrated as a component in the EA framework has demonstrated superior performance in recent years. This paper presents a comprehensive survey on integrating reinforcement learning into the evolutionary algorithm, referred to as reinforcement learning-assisted evolutionary algorithm (RL-EA). We begin with the conceptual outlines of reinforcement learning and the evolutionary algorithm. We then provide a taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. The RL-assisted procedure is divided according to the implemented functions including solution generation, learnable objective function, algorithm/operator/sub-population selection, parameter adaptation, and other strategies. Additionally, different attribute settings of RL in RL-EA are discussed. In the applications of RL-EA section, we also demonstrate the excellent performance of RL-EA on several benchmarks and a range of public datasets to facilitate a quick comparative study. Finally, we analyze potential directions for future research.

Figures (16)

• Figure 1: Overview of the number of RL-EA related studies by year
• Figure 2: Flowchart of a generic evolutionary algorithm
• Figure 3: Information interaction between agent and environment in RL
• Figure 4: The taxonomy of RL-EA in the survey
• Figure 5: Direct integration of RL and EA