Dissecting the Impact of Model Misspecification in Data-driven Optimization
Adam N. Elmachtoub, Henry Lam, Haixiang Lan, Haofeng Zhang
TL;DR
The paper investigates how model misspecification affects data-driven optimization by comparing ETO and IEO at finite-sample scales. It introduces higher-order regret expansions and uses Berry-Esseen bounds to derive sharp, instance-specific tail-regret results. A universal advantage for IEO appears in the top two regret terms under misspecification, while ETO may be preferable when the model is nearly well-specified due to lower estimation variability in the second-order term. These findings offer practical guidance for choosing estimation-planning pipelines in prescriptive ML, especially when model misspecification is non-negligible.
Abstract
Data-driven optimization aims to translate a machine learning model into decision-making by optimizing decisions on estimated costs. Such a pipeline can be conducted by fitting a distributional model which is then plugged into the target optimization problem. While this fitting can utilize traditional methods such as maximum likelihood, a more recent approach uses estimation-optimization integration that minimizes decision error instead of estimation error. Although intuitive, the statistical benefit of the latter approach is not well understood yet is important to guide the prescriptive usage of machine learning. In this paper, we dissect the performance comparisons between these approaches in terms of the amount of model misspecification. In particular, we show how the integrated approach offers a ``universal double benefit'' on the top two dominating terms of regret when the underlying model is misspecified, while the traditional approach can be advantageous when the model is nearly well-specified. Our comparison is powered by finite-sample tail regret bounds that are derived via new higher-order expansions of regrets and the leveraging of a recent Berry-Esseen theorem.