Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms

Pengyi Li, Jianye Hao, Hongyao Tang, Xian Fu, Yan Zheng, Ke Tang

TL;DR

This survey offers a comprehensive overview of the diverse research branches in ERL, systematically summarize the recent advancements in related algorithms and identify three primary research directions: 1) EA-assisted optimization of RL; 2) RL-assisted optimization of EA; and 3) synergistic optimization of EA and RL.

Abstract

Evolutionary Reinforcement Learning (ERL), which integrates Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for optimization, has demonstrated remarkable performance advancements. By fusing both approaches, ERL has emerged as a promising research direction. This survey offers a comprehensive overview of the diverse research branches in ERL. Specifically, we systematically summarize recent advancements in related algorithms and identify three primary research directions: EA-assisted Optimization of RL, RL-assisted Optimization of EA, and synergistic optimization of EA and RL. Following that, we conduct an in-depth analysis of each research direction, organizing multiple research branches. We elucidate the problems that each branch aims to tackle and how the integration of EAs and RL addresses these challenges. In conclusion, we discuss potential challenges and prospective future research directions across various research directions. To facilitate researchers in delving into ERL, we organize the algorithms and codes involved on https://github.com/yeshenpy/Awesome-Evolutionary-Reinforcement-Learning.

Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms

TL;DR

This survey offers a comprehensive overview of the diverse research branches in ERL, systematically summarize the recent advancements in related algorithms and identify three primary research directions: 1) EA-assisted optimization of RL; 2) RL-assisted optimization of EA; and 3) synergistic optimization of EA and RL.

Abstract

Evolutionary Reinforcement Learning (ERL), which integrates Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for optimization, has demonstrated remarkable performance advancements. By fusing both approaches, ERL has emerged as a promising research direction. This survey offers a comprehensive overview of the diverse research branches in ERL. Specifically, we systematically summarize recent advancements in related algorithms and identify three primary research directions: EA-assisted Optimization of RL, RL-assisted Optimization of EA, and synergistic optimization of EA and RL. Following that, we conduct an in-depth analysis of each research direction, organizing multiple research branches. We elucidate the problems that each branch aims to tackle and how the integration of EAs and RL addresses these challenges. In conclusion, we discuss potential challenges and prospective future research directions across various research directions. To facilitate researchers in delving into ERL, we organize the algorithms and codes involved on https://github.com/yeshenpy/Awesome-Evolutionary-Reinforcement-Learning.
Paper Structure (9 sections, 4 figures, 4 tables)

This paper contains 9 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Evolutionary Algorithms
  4. Reinforcement Learning
  5. Problem Definition
  6. EA-assisted Optimization of RL
  7. RL-assisted Optimization of EA
  8. Synergistic Optimization of EA and RL
  9. Conclusion

Figures (4)

  • Figure 1: Three major research directions in the ERL field. Each direction comprises multiple research branches.
  • Figure 2: Evolutionary Algorithm optimization process. Diamond-shaped blocks represent the actions taken by the algorithm, while rectangular blocks represent instances generated by the actions.
  • Figure 3: Reinforcement Learning process.
  • Figure 4: Schematic of Four Integration Approaches: (a) EA-assisted Optimization of RL. RL conducts search and improvement of the solution, with EAs playing a supporting role; EAs cannot independently optimize solutions. (b) RL-assisted Optimization of EA. EAs conduct search and improvement of the solution, with RL playing a supporting role; RL cannot independently optimize solutions. (c) and (d) Synergistic Optimization of EA and RL.