From Data to Uncertainty Sets: a Machine Learning Approach
Dimitris Bertsimas, Benjamin Boucher
TL;DR
This paper addresses the challenge of enforcing constraints when model-predicted parameters are uncertain. It introduces a loss-based uncertainty set that adapts to covariates via a trained predictor, and establishes probabilistic guarantees for constraint satisfaction. The approach generalizes ellipsoidal uncertainty sets, remains tractable under concave constraint functions, and offers stronger guarantees, including tailored bounds for mean-squared-error based regression with potential variance prediction. Empirical results on Newsvendor, Portfolio Optimization, and Shortest Path demonstrate substantially smaller uncertainty radii and improved objective values and regret relative to baselines. The method provides a practical, theoretically grounded way to incorporate ML uncertainty into prescriptive optimization with covariate information.
Abstract
Existing approaches of prescriptive analytics -- where inputs of an optimization model can be predicted by leveraging covariates in a machine learning model -- often attempt to optimize the mean value of an uncertain objective. However, when applied to uncertain constraints, these methods rarely work because satisfying a crucial constraint in expectation may result in a high probability of violation. To remedy this, we leverage robust optimization to protect a constraint against the uncertainty of a machine learning model's output. To do so, we design an uncertainty set based on the model's loss function. Intuitively, this approach attempts to minimize the uncertainty around a prediction. Extending guarantees from the robust optimization literature, we derive strong guarantees on the probability of violation. On synthetic computational experiments, our method requires uncertainty sets with radii up to one order of magnitude smaller than those of other approaches.