Solving Expensive Optimization Problems in Dynamic Environments with Meta-learning

TL;DR

The paper addresses expensive dynamic optimization where objectives change over time under strict evaluation budgets. It proposes MLO, a flexible framework that combines a gradient-based meta-learning component to learn a good initial surrogate parameter $\theta^{ml}$ from past environments with an adaptation component that quickly tunes the surrogate on a new environment using few-shot data $\mathcal{D}_t$, applicable to both data-driven evolutionary optimization and Bayesian optimization. The key contributions include the integration of meta-learning with plug-in differentiable surrogates (e.g., GPR, NN) and classical solvers (CMA-ES, PSO, DE), a formal meta-learning procedure resembling MAML, and a thorough complexity and empirical study demonstrating improved solution quality and efficiency across dynamic benchmarks. The framework enables rapid, resource-efficient tracking of moving optima in changing environments, offering practical impact for real-world expensive optimization problems where data are scarce and changes occur in discrete steps.

Abstract

Dynamic environments pose great challenges for expensive optimization problems, as the objective functions of these problems change over time and thus require remarkable computational resources to track the optimal solutions. Although data-driven evolutionary optimization and Bayesian optimization (BO) approaches have shown promise in solving expensive optimization problems in static environments, the attempts to develop such approaches in dynamic environments remain rarely unexplored. In this paper, we propose a simple yet effective meta-learning-based optimization framework for solving expensive dynamic optimization problems. This framework is flexible, allowing any off-the-shelf continuously differentiable surrogate model to be used in a plug-in manner, either in data-driven evolutionary optimization or BO approaches. In particular, the framework consists of two unique components: 1) the meta-learning component, in which a gradient-based meta-learning approach is adopted to learn experience (effective model parameters) across different dynamics along the optimization process. 2) the adaptation component, where the learned experience (model parameters) is used as the initial parameters for fast adaptation in the dynamic environment based on few shot samples. By doing so, the optimization process is able to quickly initiate the search in a new environment within a strictly restricted computational budget. Experiments demonstrate the effectiveness of the proposed algorithm framework compared to several state-of-the-art algorithms on common benchmark test problems under different dynamic characteristics.

Figures (12)

• Figure 1: Illustrative example of meta-learning working for two tasks.
• Figure 2: Flowchart of our proposed meta-learning-based optimization approach for expensive dynamic optimization.
• Figure 3: Loss function of the mean errors between the true objective functions and the best objective values along with a confidence level over time at different dimensions with $h_{sev}=7.0$ and $x_{sev}=5.0$ when comparing MLDDEO by using GPR and MLBO with other peer algorithms, respectively.
• Figure 4: Box plots of Scott-Knott test ranks of $E_{BBC}$ obtained by the proposed algorithm instances and the corresponding peer algorithms with $h_{sev}=7.0$ and $x_{sev}=5.0$ (the smaller rank is, the better performance achieved).
• Figure 5: Percentage of $A_{12}$ effect size of $E_{BBC}$ with $h_{sev}=7.0$ and $x_{sev}=5.0$ when comparing MLDDEO or MLBO with corresponding state-of-the-art peer algorithms.

Theorems & Definitions (2)

• Remark 1
• Remark 2