Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sliced gradient-enhanced Kriging for high-dimensional function approximation

Kai Cheng, Ralf Zimmermann

TL;DR

SGE-Kriging is developed, which replaces the original high-dimensional hyper-parameter tuning problem with a low-dimensional counterpart by learning the relationship between the hyper-parameters and the derivative-based global sensitivity indices, and features an accuracy and robustness that is comparable to the standard one but comes at much less training costs.

Abstract

Gradient-enhanced Kriging (GE-Kriging) is a well-established surrogate modelling technique for approximating expensive computational models. However, it tends to get impractical for high-dimensional problems due to the size of the inherent correlation matrix and the associated high-dimensional hyper-parameter tuning problem. To address these issues, a new method, called sliced GE-Kriging (SGE-Kriging), is developed in this paper for reducing both the size of the correlation matrix and the number of hyper-parameters. We first split the training sample set into multiple slices, and invoke Bayes' theorem to approximate the full likelihood function via a sliced likelihood function, in which multiple small correlation matrices are utilized to describe the correlation of the sample set rather than one large one. Then, we replace the original high-dimensional hyper-parameter tuning problem with a low-dimensional counterpart by learning the relationship between the hyper-parameters and the derivative-based global sensitivity indices. The performance of SGE-Kriging is finally validated by means of numerical experiments with several benchmarks and a high-dimensional aerodynamic modeling problem. The results show that the SGE-Kriging model features an accuracy and robustness that is comparable to the standard one but comes at much less training costs. The benefits are most evident for high-dimensional problems with tens of variables.

Sliced gradient-enhanced Kriging for high-dimensional function approximation

TL;DR

SGE-Kriging is developed, which replaces the original high-dimensional hyper-parameter tuning problem with a low-dimensional counterpart by learning the relationship between the hyper-parameters and the derivative-based global sensitivity indices, and features an accuracy and robustness that is comparable to the standard one but comes at much less training costs.

Abstract

Gradient-enhanced Kriging (GE-Kriging) is a well-established surrogate modelling technique for approximating expensive computational models. However, it tends to get impractical for high-dimensional problems due to the size of the inherent correlation matrix and the associated high-dimensional hyper-parameter tuning problem. To address these issues, a new method, called sliced GE-Kriging (SGE-Kriging), is developed in this paper for reducing both the size of the correlation matrix and the number of hyper-parameters. We first split the training sample set into multiple slices, and invoke Bayes' theorem to approximate the full likelihood function via a sliced likelihood function, in which multiple small correlation matrices are utilized to describe the correlation of the sample set rather than one large one. Then, we replace the original high-dimensional hyper-parameter tuning problem with a low-dimensional counterpart by learning the relationship between the hyper-parameters and the derivative-based global sensitivity indices. The performance of SGE-Kriging is finally validated by means of numerical experiments with several benchmarks and a high-dimensional aerodynamic modeling problem. The results show that the SGE-Kriging model features an accuracy and robustness that is comparable to the standard one but comes at much less training costs. The benefits are most evident for high-dimensional problems with tens of variables.
Paper Structure (19 sections, 54 equations, 22 figures, 7 tables)

This paper contains 19 sections, 54 equations, 22 figures, 7 tables.

Table of Contents

  1. Introduction
  2. A review of gradient-enhanced Kriging framework
  3. Gradient-enhanced Kriging model
  4. Maximum likelihood estimation
  5. Correlation function
  6. Sliced gradient-enhanced Kriging
  7. Derivative-based global sensitivity analysis
  8. Sliced likelihood function
  9. Cost-reduced hyper-parameter tuning
  10. Numerical examples
  11. One-dimensional example
  12. Two-dimensional example
  13. 8-dimensional Borehole function
  14. 30-dimensional Rosenbrock function
  15. High-dimensional Dixon-Price function
  16. ...and 4 more sections

Figures (22)

  • Figure 1: Biquadratic spline correlation function with different hyper-parameter $\theta$.
  • Figure 2: A two dimensional example of splitting the whole training sample set (left) into six slices (right) along the $x_1$-coordinate direction
  • Figure 3: Computational cost ratios for computing the Cholesky decomposition of the correlation matrices in GE-Kriging and SGE-Kriging.
  • Figure 4: The true response in Example \ref{['sec:Ex1']}.
  • Figure 5: Comparison of likelihood functions of GE-Kriging and SGE-Kriging with $m=5$ using cubic spline correlation function (left) and biquadratic spline correlation function (right) for Example \ref{['sec:Ex1']}.
  • ...and 17 more figures