Warm Starting of CMA-ES for Contextual Optimization Problems

TL;DR

The paper tackles contextual optimization where the objective $f(oldsymbol{x},\boldsymbol{\alpha})$ depends on context $\boldsymbol{\alpha}$ and seeks context-specific optima $\boldsymbol{x}^*(\boldsymbol{\alpha})$. It introduces CMA-ES-CWS, which learns a predictive distribution of the optimal solution for a new context from past context–solution pairs using a multi-output Gaussian process regression with linear model of coregionalization, and warm-starts CMA-ES from this predictive distribution. Empirical results on transformed benchmark functions and robot-control tasks demonstrate that CMA-ES-CWS outperforms existing contextual CMA-ES and WS-CMA-ES, especially under nonlinear and noisy context shifts, with gains increasing as more past optimization data are used. The work highlights the practical impact of context-aware warm starting in evolutionary optimization and points to future extensions to discrete/mixed problems and improved covariance initialization.

Abstract

Several practical applications of evolutionary computation possess objective functions that receive the design variables and externally given parameters. Such problems are termed contextual optimization problems. These problems require finding the optimal solutions corresponding to the given context vectors. Existing contextual optimization methods train a policy model to predict the optimal solution from context vectors. However, the performance of such models is limited by their representation ability. By contrast, warm starting methods have been used to initialize evolutionary algorithms on a given problem using the optimization results on similar problems. Because warm starting methods do not consider the context vectors, their performances can be improved on contextual optimization problems. Herein, we propose a covariance matrix adaptation evolution strategy with contextual warm starting (CMA-ES-CWS) to efficiently optimize the contextual optimization problem with a given context vector. The CMA-ES-CWS utilizes the optimization results of past context vectors to train the multivariate Gaussian process regression. Subsequently, the CMA-ES-CWS performs warm starting for a given context vector by initializing the search distribution using posterior distribution of the Gaussian process regression. The results of the numerical simulation suggest that CMA-ES-CWS outperforms the existing contextual optimization and warm starting methods.

Figures (6)

• Figure 1: Transitions of best evaluation values on the sphere, Rosenbrock, and Easom functions. We plot the medians and interquartile ranges. Dash lines show the evaluation values for outputs of the policy model in contextual CMA-ES and multi-output GPR in CMA-ES-CWS. Note that the evaluation values less than $10^{-8}$ are not plotted.
• Figure 2: Transitions of the best evaluation values with various past optimization results $M_\mathrm{prev}$. We plotted the medians and interquartile ranges of 20 trials on the sphere function.
• Figure 3: FetchPush-v2
• Figure 4: Result of algorithm comparison on FetchPush-v2
• Figure 5: FetchSlide-v2