Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Marginal Probability-Based Integer Handling for CMA-ES Tackling Single-and Multi-Objective Mixed-Integer Black-Box Optimization

Ryoki Hamano, Shota Saito, Masahiro Nomura, Shinichi Shirakawa

TL;DR

This work tackles mixed-integer black-box optimization (MI-BBO) with CMA-ES, identifying stagnation caused by naive discretization of integer variables. It introduces CMA-ES with Margin, which lower-bounds the marginal probabilities of generating integer variables via a diagonal affine transform, preserving CMA-ES invariances and enabling continued exploration. The approach extends to multi-objective optimization (MO-MI-BBO) as MO-CMA-ES with Margin, showing improved Pareto-front coverage and robustness across high-dimensional settings, especially with small population sizes. Empirical results demonstrate superior robustness and efficiency over prior mixed-integer CMA-ES variants, with a practical default margin parameter $\alpha$ around $\alpha = (N\lambda)^{-1}$. The method offers a general, principled way to mitigate variable fixation and can be applied to both single- and multi-objective problems in CIM- and EI-based CMA-ES frameworks.

Abstract

This study targets the mixed-integer black-box optimization (MI-BBO) problem where continuous and integer variables should be optimized simultaneously. The CMA-ES, our focus in this study, is a population-based stochastic search method that samples solution candidates from a multivariate Gaussian distribution (MGD), which shows excellent performance in continuous BBO. The parameters of MGD, mean and (co)variance, are updated based on the evaluation value of candidate solutions in the CMA-ES. If the CMA-ES is applied to the MI-BBO with straightforward discretization, however, the variance corresponding to the integer variables becomes much smaller than the granularity of the discretization before reaching the optimal solution, which leads to the stagnation of the optimization. In particular, when binary variables are included in the problem, this stagnation more likely occurs because the granularity of the discretization becomes wider, and the existing integer handling for the CMA-ES does not address this stagnation. To overcome these limitations, we propose a simple integer handling for the CMA-ES based on lower-bounding the marginal probabilities associated with the generation of integer variables in the MGD. The numerical experiments on the MI-BBO benchmark problems demonstrate the efficiency and robustness of the proposed method. Furthermore, in order to demonstrate the generality of the idea of the proposed method, in addition to the single-objective optimization case, we incorporate it into multi-objective CMA-ES and verify its performance on bi-objective mixed-integer benchmark problems.

Marginal Probability-Based Integer Handling for CMA-ES Tackling Single-and Multi-Objective Mixed-Integer Black-Box Optimization

TL;DR

This work tackles mixed-integer black-box optimization (MI-BBO) with CMA-ES, identifying stagnation caused by naive discretization of integer variables. It introduces CMA-ES with Margin, which lower-bounds the marginal probabilities of generating integer variables via a diagonal affine transform, preserving CMA-ES invariances and enabling continued exploration. The approach extends to multi-objective optimization (MO-MI-BBO) as MO-CMA-ES with Margin, showing improved Pareto-front coverage and robustness across high-dimensional settings, especially with small population sizes. Empirical results demonstrate superior robustness and efficiency over prior mixed-integer CMA-ES variants, with a practical default margin parameter around . The method offers a general, principled way to mitigate variable fixation and can be applied to both single- and multi-objective problems in CIM- and EI-based CMA-ES frameworks.

Abstract

This study targets the mixed-integer black-box optimization (MI-BBO) problem where continuous and integer variables should be optimized simultaneously. The CMA-ES, our focus in this study, is a population-based stochastic search method that samples solution candidates from a multivariate Gaussian distribution (MGD), which shows excellent performance in continuous BBO. The parameters of MGD, mean and (co)variance, are updated based on the evaluation value of candidate solutions in the CMA-ES. If the CMA-ES is applied to the MI-BBO with straightforward discretization, however, the variance corresponding to the integer variables becomes much smaller than the granularity of the discretization before reaching the optimal solution, which leads to the stagnation of the optimization. In particular, when binary variables are included in the problem, this stagnation more likely occurs because the granularity of the discretization becomes wider, and the existing integer handling for the CMA-ES does not address this stagnation. To overcome these limitations, we propose a simple integer handling for the CMA-ES based on lower-bounding the marginal probabilities associated with the generation of integer variables in the MGD. The numerical experiments on the MI-BBO benchmark problems demonstrate the efficiency and robustness of the proposed method. Furthermore, in order to demonstrate the generality of the idea of the proposed method, in addition to the single-objective optimization case, we incorporate it into multi-objective CMA-ES and verify its performance on bi-objective mixed-integer benchmark problems.
Paper Structure (38 sections, 1 theorem, 32 equations, 7 figures, 2 tables, 3 algorithms)

This paper contains 38 sections, 1 theorem, 32 equations, 7 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. CMA-ES and Mixed Integer Handling
  3. CMA-ES
  4. Sample and Evaluate Candidate Solutions
  5. Update Mean Vector
  6. Compute Evolution Paths
  7. Update Step-size and Covariance Matrix
  8. CMA-ES with Mixed-Integer Handling
  9. Inject Integer Mutation
  10. Modify Step-size Adaptation
  11. Preliminary Experiment: Why Is It Difficult to Optimize Binary Variables with CMA-ES Introducing Integer Mutations?
  12. Settings
  13. Result and Discussion
  14. Proposed Method
  15. Definition of $\textsc{Encoding}_f$ and Threshold $\ell$
  16. ...and 23 more sections

Key Result

Proposition 1

Consider the margin correction for an individual $\boldsymbol{a}^{(t)}_i$. Let $\tilde{\boldsymbol{x}}_i^{(t)}$ be the margin-corrected search point. Then, it holds

Figures (7)

  • Figure 1: Transition of each element of the mean vector and the diagonal elements of the covariance matrix on CMA-ES-IM hansen_cma-es_2011 with and without the box constraint for a typical single failed trial on 40-dimensional SphereOneMax.
  • Figure 2: Example of MGD followed by $\boldsymbol{v}$ and its marginal probability. The dashed red ellipse corresponds to the MGD before the correction, whereas the solid one corresponds to the MGD after the correction for the binary variable (left) and for the integer variable (right).
  • Figure 3: Heatmap of the success rate (top) and the median evaluation counts for successful cases (bottom) in the $N$-dimensional SphereInt function when the hyperparameter $\alpha=N^{-m}\lambda^{-n}$ of the proposed method is changed.
  • Figure 4: Heatmap of difference in median hypervolume after optimization between MO-CMA-ES with Margin and MO-CMA-ES without margin in the $N$-dimensional DSLOTZ function with $\mu = \lambda = 10~ \textrm{(top)}, 50~\textrm{(middle)}, 100 ~\textrm{(bottom)}$ when the hyperparameter $\alpha=N^{-m}\lambda^{-n}$ of the MO-CMA-ES with Margin is changed.
  • Figure 5: Heatmap of difference in median hypervolume after optimization between MO-CMA-ES with Margin and MO-CMA-ES without margin in the $N$-dimensional DSInt function with $\mu = \lambda = 10~ \textrm{(top)}, 50~\textrm{(middle)}, 100 ~\textrm{(bottom)}$ when the hyperparameter $\alpha=N^{-m}\lambda^{-n}$ of the MO-CMA-ES with Margin is changed.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Proposition 1