Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal ANOVA-based emulators of models with(out) derivatives

Matieyendou Lamboni

TL;DR

The paper develops derivative-based and derivative-free ANOVA-based emulators (Db-ANOVA) for high-dimensional models by exploiting Sobol' sensitivity indices to structure truncations and interactions. Derivative-based emulators yield dimension-free MSEs and a parametric convergence rate of $O(N^{-1})$ when cross-partial derivatives are available, while derivative-free surrogates extend to general input distributions with explicit bias and MSE bounds. It introduces stochastic surrogates, provides rigorous bias and MSE analyses under Hölder smoothness and general input distributions, and demonstrates the methods on Ishigami and Sobol' g-functions as well as a heat-diffusion PDE with stochastic initial conditions. The results show both approaches can efficiently approximate complex simulators, with derivative-based emulators offering stronger theoretical guarantees in high dimensions and derivative-free emulators providing robust performance with tractable sample requirements. The work also outlines practical truncation strategies and local emulator possibilities, pointing to future enhancements and broader applicability to empirical data.

Abstract

This paper proposes new ANOVA-based approximations of functions and emulators of high-dimensional models using either available derivatives or local stochastic evaluations of such models. Our approach makes use of sensitivity indices to design adequate structures of emulators. For high-dimensional models with available derivatives, our derivative-based emulators reach dimension-free mean squared errors (MSEs) and parametric rate of convergence (i.e., $\mathsf{O}(N^{-1})$). This approach is extended to cope with every model (without available derivatives) by deriving global emulators that account for the local properties of models or simulators. Such generic emulators enjoy dimension-free biases, parametric rates of convergence and MSEs that depend on the dimensionality. Dimension-free MSEs are obtained for high-dimensional models with particular inputs' distributions. Our emulators are also competitive in dealing with different distributions of the input variables and for selecting inputs and interactions. Simulations show the efficiency of our approach.

Optimal ANOVA-based emulators of models with(out) derivatives

TL;DR

The paper develops derivative-based and derivative-free ANOVA-based emulators (Db-ANOVA) for high-dimensional models by exploiting Sobol' sensitivity indices to structure truncations and interactions. Derivative-based emulators yield dimension-free MSEs and a parametric convergence rate of when cross-partial derivatives are available, while derivative-free surrogates extend to general input distributions with explicit bias and MSE bounds. It introduces stochastic surrogates, provides rigorous bias and MSE analyses under Hölder smoothness and general input distributions, and demonstrates the methods on Ishigami and Sobol' g-functions as well as a heat-diffusion PDE with stochastic initial conditions. The results show both approaches can efficiently approximate complex simulators, with derivative-based emulators offering stronger theoretical guarantees in high dimensions and derivative-free emulators providing robust performance with tractable sample requirements. The work also outlines practical truncation strategies and local emulator possibilities, pointing to future enhancements and broader applicability to empirical data.

Abstract

This paper proposes new ANOVA-based approximations of functions and emulators of high-dimensional models using either available derivatives or local stochastic evaluations of such models. Our approach makes use of sensitivity indices to design adequate structures of emulators. For high-dimensional models with available derivatives, our derivative-based emulators reach dimension-free mean squared errors (MSEs) and parametric rate of convergence (i.e., ). This approach is extended to cope with every model (without available derivatives) by deriving global emulators that account for the local properties of models or simulators. Such generic emulators enjoy dimension-free biases, parametric rates of convergence and MSEs that depend on the dimensionality. Dimension-free MSEs are obtained for high-dimensional models with particular inputs' distributions. Our emulators are also competitive in dealing with different distributions of the input variables and for selecting inputs and interactions. Simulations show the efficiency of our approach.

Paper Structure

This paper contains 26 sections, 15 theorems, 97 equations, 4 figures.

Table of Contents

  1. Introduction
  2. New insight into derivative-based ANOVA and emulators
  3. Full derivative-based emulators
  4. Adequate structures of emulators and truncations
  5. Derivative-free emulators of models
  6. Stochastic surrogates of functions using Db-ANOVA
  7. Statistical properties of our emulators
  8. Biases of the proposed emulators
  9. Mean squared errors
  10. Mean squared errors for every distribution of inputs
  11. Illustrations
  12. Test Functions
  13. Ishigami's Function ($d=3$)
  14. Sobol's g-Function ($d=10$)
  15. Application: Heat diffusion models with stochastic initial conditions
  16. ...and 11 more sections

Key Result

Proposition 1

Consider the main and total indices given by $S_j, \, S_{T_j}$ with $j=1, \ldots, d$. Then,

Figures (4)

  • Figure 1: Predictions versus simulated outputs (observations) for Ishigami's function.
  • Figure 2: Predictions versus simulated outputs (observations) for the g-function of type A.
  • Figure 3: Predictions versus simulated outputs (observations) for the g-function of type B.
  • Figure 4: Main indices ($\circ$) and upper-bounds of total indices ($+$) of $d=50$ inputs (left-top panel), and observations versus predictions using either derivative-based emulators (see right-top panel when including all component, and left-bottom panel otherwise) or derivative-free emulators (right-bottom panel).

Theorems & Definitions (21)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Definition 3
  • Theorem 1
  • proof
  • Lemma 1
  • Theorem 2
  • Corollary 1
  • Corollary 2
  • ...and 11 more