The Bias-Variance Tradeoff in Data-Driven Optimization: A Local Misspecification Perspective

TL;DR

The paper tackles the performance of three data-driven optimization pipelines—SAA, ETO, and IEO—under local model misspecification, bridging well-specified and misspecified settings. By modeling misspecification as a tilt $Q_t$ with perturbation direction $u(z)$ and size $t=\Theta(n^{-\alpha})$, it derives regime-dependent asymptotics that decompose decision error into bias and variance components via influence functions. In the balanced regime $(\alpha=1/2)$, a clear bias-variance-regret tradeoff emerges: SAA minimizes bias, ETO minimizes variance, and IEO lies in between, with explicit formulas for decision bias and variance provided. In severe and mild regimes, the paper shows bias or variance dominates, yielding distinct ordering of the three methods and providing conditions under which misspecification is approximately impactless. The results offer practical guidance on method selection in data-driven optimization and deepen the theoretical understanding of misspecification effects through contiguity and projection-based analyses.

Abstract

Data-driven stochastic optimization is ubiquitous in machine learning and operational decision-making problems. Sample average approximation (SAA) and model-based approaches such as estimate-then-optimize (ETO) or integrated estimation-optimization (IEO) are all popular, with model-based approaches being able to circumvent some of the issues with SAA in complex context-dependent problems. Yet the relative performance of these methods is poorly understood, with most results confined to the dichotomous cases of the model-based approach being either well-specified or misspecified. We develop the first results that allow for a more granular analysis of the relative performance of these methods under a local misspecification setting, which models the scenario where the model-based approach is nearly well-specified. By leveraging tools from contiguity theory in statistics, we show that there is a bias-variance tradeoff between SAA, IEO, and ETO under local misspecification, and that the relative importance of the bias and the variance depends on the degree of local misspecification. Moreover, we derive explicit expressions for the decision bias, which allows us to characterize (un)impactful misspecification directions, and provide further geometric understanding of the variance.

Figures (3)

• Figure 1: Local Misspecification
• Figure 2: The direction of misspecification satisfies ${u}(\bm{z})=\prod_{j=1}^{d_z} \left(z^{(j)}\right)^2$.
• Figure 3: The direction of misspecification satisfies ${u}(\bm{z})=\prod_{j=1}^{d_z} \left(z^{(j)}-3j\right)^2/2$.

Theorems & Definitions (37)

• Definition 1: Local Perturbation
• Definition 2: Three Local Misspecification Regimes
• Lemma 1
• Definition 3: Regret
• Theorem 1: Asymptotics under Balanced Misspecification
• Theorem 2: Regret Comparisons under Balanced Misspecification
• Theorem 3: Asymptotics under Severe Misspecification
• Theorem 4: Bias/Regret Comparisons under Severe Misspecification
• Theorem 5: Approximately Impactless Misspecification Direction
• Example 1