Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"Over-optimizing" for Normality: Budget-constrained Uncertainty Quantification for Contextual Decision-making

Yanyuan Wang, Xiaowei Zhang

TL;DR

This framework provides a principled way to quantify uncertainty for contextual decision-making under computational constraints, and offers practical guidance on allocating limited resources between data acquisition and optimization effort, clarifying when to prioritize additional optimization iterations over more data to ensure valid confidence intervals for conditional performance.

Abstract

We study uncertainty quantification for contextual stochastic optimization, focusing on weighted sample average approximation (wSAA), which uses machine-learned relevance weights based on covariates. Although wSAA is widely used for contextual decisions, its uncertainty quantification remains limited. In addition, computational budgets tie sample size to optimization accuracy, creating a coupling that standard analyses often ignore. We establish central limit theorems for wSAA and construct asymptotic-normality-based confidence intervals for optimal conditional expected costs. We analyze the statistical--computational tradeoff under a computational budget, characterizing how to allocate resources between sample size and optimization iterations to balance statistical and optimization errors. These allocation rules depend on structural parameters of the objective; misspecifying them can break the asymptotic optimality of the wSAA estimator. We show that ``over-optimizing'' (running more iterations than the nominal rule) mitigates this misspecification and preserves asymptotic normality, at the expense of a slight slowdown in the convergence rate of the budget-constrained estimator. The common intuition that ``more data is better'' can fail under computational constraints: increasing the sample size may worsen statistical inference by forcing fewer algorithm iterations and larger optimization error. Our framework provides a principled way to quantify uncertainty for contextual decision-making under computational constraints. It offers practical guidance on allocating limited resources between data acquisition and optimization effort, clarifying when to prioritize additional optimization iterations over more data to ensure valid confidence intervals for conditional performance.

"Over-optimizing" for Normality: Budget-constrained Uncertainty Quantification for Contextual Decision-making

TL;DR

This framework provides a principled way to quantify uncertainty for contextual decision-making under computational constraints, and offers practical guidance on allocating limited resources between data acquisition and optimization effort, clarifying when to prioritize additional optimization iterations over more data to ensure valid confidence intervals for conditional performance.

Abstract

We study uncertainty quantification for contextual stochastic optimization, focusing on weighted sample average approximation (wSAA), which uses machine-learned relevance weights based on covariates. Although wSAA is widely used for contextual decisions, its uncertainty quantification remains limited. In addition, computational budgets tie sample size to optimization accuracy, creating a coupling that standard analyses often ignore. We establish central limit theorems for wSAA and construct asymptotic-normality-based confidence intervals for optimal conditional expected costs. We analyze the statistical--computational tradeoff under a computational budget, characterizing how to allocate resources between sample size and optimization iterations to balance statistical and optimization errors. These allocation rules depend on structural parameters of the objective; misspecifying them can break the asymptotic optimality of the wSAA estimator. We show that ``over-optimizing'' (running more iterations than the nominal rule) mitigates this misspecification and preserves asymptotic normality, at the expense of a slight slowdown in the convergence rate of the budget-constrained estimator. The common intuition that ``more data is better'' can fail under computational constraints: increasing the sample size may worsen statistical inference by forcing fewer algorithm iterations and larger optimization error. Our framework provides a principled way to quantify uncertainty for contextual decision-making under computational constraints. It offers practical guidance on allocating limited resources between data acquisition and optimization effort, clarifying when to prioritize additional optimization iterations over more data to ensure valid confidence intervals for conditional performance.

Paper Structure

This paper contains 40 sections, 11 theorems, 57 equations, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Related Works
  4. Notation and Organization
  5. Problem Formulation
  6. Weighted Sample Average Approximation
  7. Basic Assumptions
  8. Statistical--Computational Tradeoffs
  9. Idealized Case: No Computational Constraints
  10. Uncertainty Quantification with Computational Constraints
  11. "Over-optimizing" for Normality
  12. Extensions
  13. Sublinearly Convergent Algorithms
  14. Superlinearly Convergent Algorithms
  15. Confidence Intervals
  16. ...and 25 more sections

Key Result

Proposition 1

Under Assumptions assump:regularity--assumption:band, as $n\to\infty$, for all $x_0 \in \mathcal{X}$, where $\mathbb G(\cdot, x_0)$ is a zero-mean Gaussian process with covariance function Here, $R_2(K) \coloneqq \int_{\mathbb R^{d_x}} K^2(u) du < \infty$.

Figures (10)

  • Figure 1: Distribution of wSAA Estimates of Optimal Conditional Expected Cost.
  • Figure 2: Collapse of Asymptotic Normality of Budget-Constrained wSAA Estimator
  • Figure 3: Benefits of Over-optimizing
  • Figure 4: Citi Bike Sharing System
  • Figure 5: "Feels Like" Temperature, Wind Speed, and Hourly Demand
  • ...and 5 more figures

Theorems & Definitions (19)

  • Remark 1
  • Proposition 1
  • Theorem 1
  • Definition 1: Linear Convergence
  • Example 1
  • Theorem 2
  • Theorem 3
  • Definition 2: Sublinear Convergence
  • Example 2
  • Theorem 4
  • ...and 9 more