Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying Distributional Input Uncertainty via Inflated Kolmogorov-Smirnov Confidence Band

Motong Chen, Henry Lam, Zhenyuan Liu

Abstract

In stochastic simulation, input uncertainty refers to the propagation of the statistical noise in calibrating input models to impact output accuracy, in addition to the Monte Carlo simulation noise. The vast majority of the input uncertainty literature focuses on estimating target output quantities that are real-valued. However, outputs of simulation models are random and real-valued targets essentially serve only as summary statistics. To provide a more holistic assessment, we study the input uncertainty problem from a distributional view, namely we construct confidence bands for the entire output distribution function. Our approach utilizes a novel test statistic whose asymptotic consists of the supremum of the sum of a Brownian bridge and a suitable mean-zero Gaussian process, which generalizes the Kolmogorov-Smirnov statistic to account for input uncertainty. Regarding implementation, we also demonstrate how to use subsampling to efficiently estimate the covariance function of the Gaussian process, thereby leading to an implementable estimation of the quantile of the test statistic and a statistically valid confidence band. Numerical results demonstrate how our new confidence bands provide valid coverage for output distributions under input uncertainty that is not achievable by conventional approaches.

Quantifying Distributional Input Uncertainty via Inflated Kolmogorov-Smirnov Confidence Band

Abstract

In stochastic simulation, input uncertainty refers to the propagation of the statistical noise in calibrating input models to impact output accuracy, in addition to the Monte Carlo simulation noise. The vast majority of the input uncertainty literature focuses on estimating target output quantities that are real-valued. However, outputs of simulation models are random and real-valued targets essentially serve only as summary statistics. To provide a more holistic assessment, we study the input uncertainty problem from a distributional view, namely we construct confidence bands for the entire output distribution function. Our approach utilizes a novel test statistic whose asymptotic consists of the supremum of the sum of a Brownian bridge and a suitable mean-zero Gaussian process, which generalizes the Kolmogorov-Smirnov statistic to account for input uncertainty. Regarding implementation, we also demonstrate how to use subsampling to efficiently estimate the covariance function of the Gaussian process, thereby leading to an implementable estimation of the quantile of the test statistic and a statistically valid confidence band. Numerical results demonstrate how our new confidence bands provide valid coverage for output distributions under input uncertainty that is not achievable by conventional approaches.
Paper Structure (29 sections, 31 theorems, 412 equations, 9 figures, 1 table, 3 algorithms)

This paper contains 29 sections, 31 theorems, 412 equations, 9 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. From Conventional to Distributional Input Uncertainty
  3. The Conventional Problem
  4. Distributional Input Uncertainty
  5. Methodology to Quantify Distributional Input Uncertainty
  6. Main Idea on Interval Construction
  7. Main Theoretical Guarantees
  8. Efficient Implementation
  9. Procedural Specifications and Covariance Function Subsampling
  10. Statistical Guarantees on Covariance Function Subsampling
  11. Main Technical Developments for Theoretical Guarantees
  12. Weak Convergence of Generalized Kolmogorov-Smirnov Statistic
  13. Covariance Function Estimation
  14. Numerical Experiments
  15. Statistical Validity and Performance Comparisons of Confidence Bands
  16. ...and 14 more sections

Key Result

Theorem 1

Suppose Assumptions balanced_data, balanced_randomness and finite_horizon_model hold. We have as $n,R\rightarrow\infty$, where $BB(\cdot)$ and $\mathbb{G}(\cdot)$ are two independent stochastic processes, $BB(\cdot)$ is the standard Brownian bridge on $[0,1]$, $\mathbb{G}(\cdot)$ is a mean-zero Gaussian process defined on $\mathbb{R}$ having bounded continuous paths almost surely and the fol where

Figures (9)

  • Figure 1: A typical instance of various confidence bands (CBs) with and without input uncertainty. The curve in both figures denotes the true output distribution function. The step functions in (a) are $U_{KS}(t)$, the empirical output distribution function, and $L_{KS}(t)$ respectively reading from top to bottom. The step functions in (b) are $U_{IU}(t)$, $U_{KS}(t)$, the empirical output distribution function, $L_{KS}(t)$ and $L_{IU}(t)$ respectively reading from top to bottom.
  • Figure 2: M/M/1 queue with minimum data size $\min n_i=500$. $\theta=1$ corresponds to the standard bootstrap instead of our subsampling bootstrap.
  • Figure 3: M/M/1 queue with minimum data size $\min n_i=1000$. $\theta=1$ corresponds to the standard bootstrap instead of our subsampling bootstrap.
  • Figure 4: M/M/1 queue with minimum data size $\min n_i=2000$. $\theta=1$ corresponds to the standard bootstrap instead of our subsampling bootstrap.
  • Figure 5: Computer communication network simulation model with minimum data size $\min n_i=500$. $\theta=1$ corresponds to the standard bootstrap instead of our subsampling bootstrap.
  • ...and 4 more figures

Theorems & Definitions (53)

  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Corollary 2
  • Theorem 3
  • Lemma 1
  • Proposition 1
  • Theorem 4
  • Corollary 3
  • Theorem 5
  • ...and 43 more