Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-dimensional mixed-categorical Gaussian processes with application to multidisciplinary design optimization for a green aircraft

Paul Saves, Youssef Diouane, Nathalie Bartoli, Thierry Lefebvre, Joseph Morlier

TL;DR

This paper tackles surrogate modeling for expensive simulations with high-dimensional mixed-categorical inputs by extending Gaussian process modeling with a dimension-reducing partial least squares framework (KPLS). It combines continuous, integer, and categorical kernels via a product formulation and introduces cross-level encoding and matrix-based PLS to dramatically reduce hyperparameters while preserving predictive fidelity, especially in homogeneous categorical (HH) and exponential HH (EHH) kernels. The approach is validated on analytic benchmarks, a cantilever beam structural model, and a multidisciplinary design optimization case for green aircraft, demonstrating improved efficiency and substantial fuel reductions in a single mission. The work advances practical MDO by enabling scalable BO with mixed inputs, and its open-source SMT implementation supports reproducibility and broader adoption in engineering design optimization.

Abstract

Recently, there has been a growing interest in mixed-categorical metamodels based on Gaussian Process (GP) for Bayesian optimization. In this context, different approaches can be used to build the mixed-categorical GP. Many of these approaches involve a high number of hyperparameters; in fact, the more general and precise the strategy used to build the GP, the greater the number of hyperparameters to estimate. This paper introduces an innovative dimension reduction algorithm that relies on partial least squares regression to reduce the number of hyperparameters used to build a mixed-variable GP. Our goal is to generalize classical dimension reduction techniques commonly used within GP (for continuous inputs) to handle mixed-categorical inputs. The good potential of the proposed method is demonstrated in both structural and multidisciplinary application contexts. The targeted applications include the analysis of a cantilever beam as well as the optimization of a green aircraft, resulting in a significant 439-kilogram reduction in fuel consumption during a single mission.

High-dimensional mixed-categorical Gaussian processes with application to multidisciplinary design optimization for a green aircraft

TL;DR

This paper tackles surrogate modeling for expensive simulations with high-dimensional mixed-categorical inputs by extending Gaussian process modeling with a dimension-reducing partial least squares framework (KPLS). It combines continuous, integer, and categorical kernels via a product formulation and introduces cross-level encoding and matrix-based PLS to dramatically reduce hyperparameters while preserving predictive fidelity, especially in homogeneous categorical (HH) and exponential HH (EHH) kernels. The approach is validated on analytic benchmarks, a cantilever beam structural model, and a multidisciplinary design optimization case for green aircraft, demonstrating improved efficiency and substantial fuel reductions in a single mission. The work advances practical MDO by enabling scalable BO with mixed inputs, and its open-source SMT implementation supports reproducibility and broader adoption in engineering design optimization.

Abstract

Recently, there has been a growing interest in mixed-categorical metamodels based on Gaussian Process (GP) for Bayesian optimization. In this context, different approaches can be used to build the mixed-categorical GP. Many of these approaches involve a high number of hyperparameters; in fact, the more general and precise the strategy used to build the GP, the greater the number of hyperparameters to estimate. This paper introduces an innovative dimension reduction algorithm that relies on partial least squares regression to reduce the number of hyperparameters used to build a mixed-variable GP. Our goal is to generalize classical dimension reduction techniques commonly used within GP (for continuous inputs) to handle mixed-categorical inputs. The good potential of the proposed method is demonstrated in both structural and multidisciplinary application contexts. The targeted applications include the analysis of a cantilever beam as well as the optimization of a green aircraft, resulting in a significant 439-kilogram reduction in fuel consumption during a single mission.
Paper Structure (22 sections, 2 theorems, 25 equations, 10 figures, 10 tables)

This paper contains 22 sections, 2 theorems, 25 equations, 10 figures, 10 tables.

Table of Contents

  1. Introduction
  2. GP for mixed-categorical inputs
  3. A mixed GP formulation
  4. The homogeneous categorical kernel
  5. KPLS for mixed-categorical inputs
  6. KPLS for continuous inputs
  7. PLS for vectors inputs
  8. KPLS for continuous variables
  9. Extension of PLS to matrix inputs with application to mixed-categorical GP
  10. A PLS framework for matrix inputs
  11. A new KPLS model for categorical variables
  12. Results and discussion
  13. Implementation details
  14. Surrogate modeling
  15. Analytic validation on a categorical cosine problem ($n= 1$, $m=0$, $l=1$ and $L_1=13$)
  16. ...and 7 more sections

Key Result

Theorem 1

Assuming that all the entries of $\hat{\Theta}$ are in $[-1,1]$ and that $G^*$ is computed using PLS as in Eq. (SMO_eq:catKPLS), the matrix $\Theta$ given by Eq. (SMO_eq:true_pls_matrix_reduction) also takes values in [-1,1].

Figures (10)

  • Figure 1: Correlation matrices and associated predictions on the cosine problem using a DoE of 98 points.
  • Figure 2: Cantilever beam problem Mixed_Paul.
  • Figure 3: Correlation matrix $R_1^{cat}$ using different choices for $\Theta_1$ for the categorical variable $\tilde{I}$ from the cantilever beam problem.
  • Figure 4: Optimization results for the Toy function CAT-EGO for 20 DoE of 5 points.
  • Figure 5: Optimization results for the Toy function CAT-EGO for 20 DoE of 10 points.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Example 1
  • Theorem 1
  • proof
  • Theorem 2
  • proof