A general framework for modeling Gaussian process with qualitative and quantitative factors

TL;DR

This work extends the latent variable-based GP approach, which maps qualitative factors into a continuous latent space, by establishing a general framework to apply standard kernel functions to continuous latent variables and introduces new covariance structures in some situations.

Abstract

Computer experiments involving both qualitative and quantitative (QQ) factors have attracted increasing attention. Gaussian process (GP) models have proven effective in this context by choosing specialized covariance functions for QQ factors. In this work, we extend the latent variable-based GP approach, which maps qualitative factors into a continuous latent space, by establishing a general framework to apply standard kernel functions to continuous latent variables. This approach provides a novel perspective for interpreting some existing GP models for QQ factors and introduces new covariance structures in some situations. The ordinal structure can be incorporated naturally and seamlessly in this framework. Furthermore, the Bayesian information criterion and leave-one-out cross-validation are employed for model selection and model averaging. The performance of the proposed method is comprehensively studied on several examples.

Figures (6)

• Figure 1: Comparison of RRMSE across different methods and kernel configurations for the beam bending, borehole, OTL circuit, and piston examples. Each boxplot summarizes the results from 30 independent runs with different training sets generated via maximin LHD. All methods were evaluated on the same set of 10,000 uniformly distributed test points.
• Figure 2: The boxplots and scatter points depict the latent vectors $z_1$ for resistance $R_f$ and current gain $\beta$, estimated by $\mathsf{Gau}^{\mathsf{multi}}_{1}$, $\mathsf{Gau}^{\mathsf{multi}}_{\mathsf{ord}}$, $\mathsf{Exp}^{\mathsf{multi}}_{1}$, and $\mathsf{Exp}^{\mathsf{multi}}_{\mathsf{ord}}$, respectively, across 30 replications in the OTL example. Points from the same replication are connected by lines. In the $R_f$ panel for $\mathsf{Exp}^{\mathsf{multi}}_{1}$, different line types indicate different ordering patterns of the estimated latent vectors: solid lines correspond to replications where the original level order is preserved, dashed lines indicate a reversal between the second and third levels, and dotted lines indicate a swap between the third and fourth levels.
• Figure 3: Comparison of RRMSE across different methods and kernel configurations for the 3D coupled finite element model for embankments, evaluated on the remaining points of the training dataset. Each boxplot summarizes results from 100 runs. In each run, the model is trained using a subset of 27, 45, or 63 design points randomly sampled from the training dataset.
• Figure 4: Comparison of RRMSE across different methods and kernel configurations for the 3D coupled finite element model for embankments, evaluated on the independent test dataset. Each boxplot summarizes results from 100 runs. In each run, the model is trained using a subset of 27, 45, or 63 design points randomly sampled from the training dataset.
• Figure 5: Comparison of RRMSE across different methods and kernel configurations for the material design example. Each boxplot reflects results from 10 resampling runs, where 200 points are used for training and 23 points for testing in each run.

Theorems & Definitions (8)

• Definition 1: Parameterization Equivalence
• Theorem 1
• Remark 1
• Proposition 1: Identifiability for linear kernel
• Proposition 2: Identifiability for isotropic kernel
• proof : Proof of Theorem \ref{['thm:general_linear']}
• proof : Proof of Proposition \ref{['prop:unique_linear']}
• proof : Proof of Proposition \ref{['prop:unique_isotropic']}