Variable Metric Evolution Strategies for High-dimensional Multi-Objective Optimization

TL;DR

The paper tackles high-dimensional multi-objective optimization by combining scalable, low-rank covariance learning with indicator-based MOEAs. It introduces two algorithms: an elitist (1+1)-LM-MA-ES and MO-LM-MA-ES, the latter integrating LM-MA-ES into the MO-CMA-ES framework with a low-rank covariance model $C = I + \sum_{i=1}^k m_i m_i^T$, achieving $O(n \log n)$ per-sample complexity. Empirical results on high-dimensional bi-objective benchmarks show that MO-LM-MA-ES not only matches but often surpasses full-rank baselines in both speed and Pareto-front quality, up to dimensions of $n=1024$. The work demonstrates that scalable, high-dimensional MOEAs are practical and effective, and provides open-source implementations and benchmarks to foster reproducibility and further research.

Abstract

We design a class of variable metric evolution strategies well suited for high-dimensional problems. We target problems with many variables, not (necessarily) with many objectives. The construction combines two independent developments: efficient algorithms for scaling covariance matrix adaptation to high dimensions, and evolution strategies for multi-objective optimization. In order to design a specific instance of the class we first develop a (1+1) version of the limited memory matrix adaptation evolution strategy and then use an established standard construction to turn a population thereof into a state-of-the-art multi-objective optimizer with indicator-based selection. The method compares favorably to adaptation of the full covariance matrix.

Figures (4)

• Figure 1: Median fitness difference $f(x) - f^*$ over number of function evaluations for (1+1)-LM-MA-ES in dimensions 128 (red); 256 (orange), 512 (yellow), 1024 (green), 2048 (blue), and 4096 (magenta). Both axes use a logarithmic scale.
• Figure 2: Dominated hypervolume of MO-LM-MA-ES on all nine problems in dimensions 128 (red), 256 (yellow), 512 (green) and 1024 (blue). The red dashed curve refers to adaptation of the full covariance matrix in dimension 128. The horizontal axis is the number of function evaluations, normalized (divided) by population size and problem dimension.
• Figure 3: Convergence of the population to the optimal $\mu$-distribution. The black dots indicate the optimal $\mu$-distribution for $\mu=20$. It can be observed for both problems that the population (in objective space) converges to the optimal configuration.
• Figure 4: Log-log-plot of the hypervolume gap over the number of function evaluations. The red and solid curve corresponds to the low-rank algorithm, while the black dashed curve corresponds to adapting a full-rank covariance matrix.