Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust estimation with latin hypercube sampling: a central limit theorem for Z-estimators

Faouzi Hakimi

TL;DR

This work extends Latin Hypercube Sampling (LHS) theory from the empirical mean to the broader class of Z-estimators, which are zeros of an empirical mean function. It proves that LHS reduces the asymptotic variance of Z-estimators relative to IID sampling and establishes a Central Limit Theorem under LHS, under suitable regularity. The framework is applied to Generalized Linear Models (GLMs), including Poisson regression, with a numerical demonstration showing improved variance, mean squared error, and asymptotic normality as sample size grows. The results provide a theoretical foundation for efficient parameter estimation and metamodeling in computer experiments, with avenues for relaxing assumptions and extending to other estimator classes and dependent designs.

Abstract

Latin hypercube sampling (LHS) is a widely used stratified sampling method in computer experiments. In this work, we extend the existing convergence results for the sample mean under LHS to the broader class of $Z$-estimators, estimators defined as the zeros of a sample mean function. We derive the asymptotic variance of these estimators and demonstrate that it is smaller when using LHS compared to traditional independent and identically distributed (i.i.d.) sampling. Furthermore, we establish a Central Limit Theorem for $Z$-estimators under LHS, providing a theoretical foundation for its improved efficiency.

Robust estimation with latin hypercube sampling: a central limit theorem for Z-estimators

TL;DR

This work extends Latin Hypercube Sampling (LHS) theory from the empirical mean to the broader class of Z-estimators, which are zeros of an empirical mean function. It proves that LHS reduces the asymptotic variance of Z-estimators relative to IID sampling and establishes a Central Limit Theorem under LHS, under suitable regularity. The framework is applied to Generalized Linear Models (GLMs), including Poisson regression, with a numerical demonstration showing improved variance, mean squared error, and asymptotic normality as sample size grows. The results provide a theoretical foundation for efficient parameter estimation and metamodeling in computer experiments, with avenues for relaxing assumptions and extending to other estimator classes and dependent designs.

Abstract

Latin hypercube sampling (LHS) is a widely used stratified sampling method in computer experiments. In this work, we extend the existing convergence results for the sample mean under LHS to the broader class of -estimators, estimators defined as the zeros of a sample mean function. We derive the asymptotic variance of these estimators and demonstrate that it is smaller when using LHS compared to traditional independent and identically distributed (i.i.d.) sampling. Furthermore, we establish a Central Limit Theorem for -estimators under LHS, providing a theoretical foundation for its improved efficiency.

Paper Structure

This paper contains 9 sections, 7 theorems, 20 equations, 8 figures.

Table of Contents

  1. Introduction
  2. General notations, definitions, and properties of LHS
  3. Definitions and properties on Z-estimators
  4. Z-estimators under LHS
  5. Application: parameters estimation of Generalized Linear Models (GLM)
  6. Definitions and main properties on GLM
  7. $Z$-estimation of GLM parameters under LHS
  8. Numerical example: a Poisson regression under LHS
  9. Conclusion and prospects

Key Result

proposition 1

Let $g: [0,1]^d \rightarrow \mathbb{R}^q$ with $d,q \in \mathbb{N}^{*}$ be a measurable function such that $\mathbb{E}(|| g (\mathbf{X}) || )< +\infty$. Denote where $\bm{x}^{(i)}, i \in \llbracket 1,n \rrbracket$ is such that $\bm{\mathrm{X}}^{LHS} = \left( \bm{x}^{(1)},\ldots, \bm{x}^{(n)} \right)^T$ with $\bm{\mathrm{X}}^{LHS}$ being defined using Equation eq:Sample_LHS. Then, $G^{LHS}_n$ is a

Figures (8)

  • Figure 1: Three schematic examples of LHS designs with dimension $d = 2$ and size $n = 4$.
  • Figure 2: Average estimation variances of $(\theta_{0,1}, \ldots \theta_{0,9})^T$ according to the sampling size $n$ for IID and LHS designs (decimal logarithmic scale).
  • Figure 3: Average estimation square bias of $(\theta_{0,1}, \ldots \theta_{0,9})^T$ according to the sampling size $n$ for IID and LHS designs (decimal logarithmic scale).
  • Figure 4: Average estimation MSE of $(\theta_{0,1}, \ldots \theta_{0,9})^T$ according to the sampling size $n$ for IID and LHS designs (decimal logarithmic scale).
  • Figure 5: Evolution of measured normalized variances of $\theta_{0,1}, \ldots \theta_{0,9}$ according to the sampling size $n$ for IID and LHS designs (decimal logarithmic scale).
  • ...and 3 more figures

Theorems & Definitions (7)

  • proposition 1
  • proposition 2
  • theorem 1
  • proposition 3
  • theorem 2
  • proposition 4
  • theorem 3