Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimization

TL;DR

This work tackles large-scale black-box optimization by integrating learning-based cooperative coevolution with CMA-ES. By formulating decomposition strategy selection as an MDP and training a PPO-based policy, the approach dynamically chooses how to partition variables, reducing dependence on expert heuristics and avoiding extra function evaluations for decomposition. Empirical results on CEC 2013 LSGO and BNS show that LCC-CMAES achieves better optimization performance with lower resource costs and exhibits transferable behavior to unseen problems, including neuroevolution tasks. The combination of MetaBBO-inspired learning, principled state design, and lightweight Actor–Critic networks positions LCC-CMAES as a robust, scalable solution for demanding LSGO challenges.

Abstract

Recent research in Cooperative Coevolution~(CC) have achieved promising progress in solving large-scale global optimization problems. However, existing CC paradigms have a primary limitation in that they require deep expertise for selecting or designing effective variable decomposition strategies. Inspired by advancements in Meta-Black-Box Optimization, this paper introduces LCC, a pioneering learning-based cooperative coevolution framework that dynamically schedules decomposition strategies during optimization processes. The decomposition strategy selector is parameterized through a neural network, which processes a meticulously crafted set of optimization status features to determine the optimal strategy for each optimization step. The network is trained via the Proximal Policy Optimization method in a reinforcement learning manner across a collection of representative problems, aiming to maximize the expected optimization performance. Extensive experimental results demonstrate that LCC not only offers certain advantages over state-of-the-art baselines in terms of optimization effectiveness and resource consumption, but it also exhibits promising transferability towards unseen problems.

Figures (5)

• Figure 1: The core idea of LCC-CMAES.
• Figure 2: The overall structure of LCC.
• Figure 3: The Neural Network workflow for $\pi_{\theta}$ (Actor) and $v_{\phi}$(Critic).
• Figure 4: Comparison with a 3E6 budget in CEC2013LSGO.
• Figure 5: The ablation study on state features and reward designs.