Surrogate-based categorical neighborhoods for mixed-variable blackbox optimization

Abstract

In simulation-based engineering, design choices are often obtained following the optimization of complex blackbox models. These models frequently involve mixed-variable domains with quantitative and categorical variables. Unlike quantitative variables, categorical variables lack an inherent structure, which makes them difficult to handle, especially in the presence of constraints. This work proposes a systematic approach to structure and model categorical variables in constrained mixed-variable blackbox optimization. Surrogate models of the objective and constraint functions are used to induce problem-specific categorical distances. From these distances, surrogate-based neighborhoods are constructed using notions of dominance from bi-objective optimization, jointly accounting for information from both the objective and the constraint functions. This study addresses the lack of automatic and constraint-aware categorical neighborhood construction in mixed-variable blackbox optimization. As a proof of concept, these neighborhoods are employed within CatMADS, an extension of the MADS algorithm for categorical variables. The surrogate models are Gaussian processes, and the resulting method is called CatMADS-GP. The method is benchmarked on the Cat-Suite collection of 60 mixed-variable optimization problems and compared against state-of-the-art solvers. Data profiles indicate that CatMADS-GP achieves superior performance for both unconstrained and constrained problems.

Figures (10)

• Figure 1: A motivating example where the incumbent $\bm{x}_{(k)}$ has objective value $f(\bm{x}_{(k)})=3$, while the global minimum $\bm{x}_{\star}$ has $f(\bm{x}_{\star})=1$. The neighborhood on the left selects purple based strictly on proximity with the objective. The neighborhood on the right selects green based on proximity of both the constraints and the objective.
• Figure 2: Statistical analysis of feasibility per categorical component in the piece machining problem.
• Figure 3: Visualization of a unconstrained neighborhood equivalently constructed in a Hilbert space.
• Figure 4: Ordering components with ranking functions. (\ref{['subfig:rank1']})-(\ref{['subfig:rank3']}) three steps for constrained problems. (\ref{['subfig:rank_unconstrained']}) single step for unconstrained problems. The component $\bm{u}$ is located at the origin and is not displayed.
• Figure 5: Ordering of categorical components with the surrogate-based neighborhood in the mechanical-part design problem. The incumbent $\bm{u}=\left(\text{\sc b}\xspace, \text{\sc wood}\xspace, \text{\sc circle}\xspace\right)$ is at the origin and it is not displayed.

Theorems & Definitions (3)

• Definition 1: Ordering relation
• Definition 2: Surrogate-based neighborhood
• Definition 3: Mixed-variable surrogate-based neighborhood