Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective
Adam N. Elmachtoub, Henry Lam, Haofeng Zhang, Yunfan Zhao
TL;DR
The paper analyzes three data-driven optimization paradigms—Sample Average Approximation (SAA), Estimate-Then-Optimize (ETO), and Integrated-Estimation-Optimization (IEO)—in a stochastic setting with parametric distribution modeling. It establishes that, under a well-specified model class and large sample sizes, the regret distribution satisfies a first-order stochastic dominance: G^{ETO} \preceq_{st} G^{IEO} \preceq_{st} G^{SAA}, with ETO asymptotically most favorable due to the Cramer–Rao bound, while SAA is worst. When the model is misspecified, the ordering reverses: the regrets of ETO and IEO no longer vanish, and the dominance relations invert, implying SAA or alternative baselines may outperform under misspecification. The results extend to constrained and contextual stochastic optimization, employing KKT conditions, projection arguments, and delta-method-based normal approximations to derive analogous dominance results. Across synthetic examples (newsvendor, portfolio optimization) and real-data experiments, the findings hold in finite-sample regimes and reveal when simpler ETO can be preferable to the integrated approach, especially in well-specified, data-rich contexts. These insights inform practical methodology by clarifying when the added computational burden of IEO yields meaningful gains versus when estimation-then-optimization suffices.
Abstract
In data-driven stochastic optimization, model parameters of the underlying distribution need to be estimated from data in addition to the optimization task. Recent literature considers integrating the estimation and optimization processes by selecting model parameters that lead to the best empirical objective performance. This integrated approach, which we call integrated-estimation-optimization (IEO), can be readily shown to outperform simple estimate-then-optimize (ETO) when the model is misspecified. In this paper, we show that a reverse behavior appears when the model class is well-specified and there is sufficient data. Specifically, for a general class of nonlinear stochastic optimization problems, we show that simple ETO outperforms IEO asymptotically when the model class covers the ground truth, in the strong sense of stochastic dominance of the regret. Namely, the entire distribution of the regret, not only its mean or other moments, is always better for ETO compared to IEO. Our results also apply to constrained, contextual optimization problems where the decision depends on observed features. Whenever applicable, we also demonstrate how standard sample average approximation (SAA) performs the worst when the model class is well-specified in terms of regret, and best when it is misspecified. Finally, we provide experimental results to support our theoretical comparisons and illustrate when our insights hold in finite-sample regimes and under various degrees of misspecification.