Learning Joint Models of Prediction and Optimization

TL;DR

This work reframes Predict-Then-Optimize (PtO) by introducing Learning to Optimize from Features (LtOF), which directly maps observable features to optimal decision solutions instead of predicting problem coefficients. By leveraging established Learn-to-Optimize (LtO) techniques (LD, PDL, DC3), LtOF handles hard optimization forms, including nonconvex components, with fast, differentiable inference and reduced reliance on bespoke backpropagation through the solver. The authors demonstrate that LtOF can outperform traditional two-stage and, in many cases, End-to-End PtO (EPO) baselines across convex QP, nonconvex QP, and nonconvex AC-OPF problems, while also offering substantial efficiency gains. A key contribution is the formalization of LtOF as a PtO framework that bypasses distribution-shift issues inherent in proxy-based EPO, enabling robust, real-time decision-making in complex optimization tasks. The results suggest LtOF as a versatile and scalable alternative for data-driven decision-making in settings with challenging optimization components and limited derivative information.

Abstract

The Predict-Then-Optimize framework uses machine learning models to predict unknown parameters of an optimization problem from exogenous features before solving. This setting is common to many real-world decision processes, and recently it has been shown that decision quality can be substantially improved by solving and differentiating the optimization problem within an end-to-end training loop. However, this approach requires significant computational effort in addition to handcrafted, problem-specific rules for backpropagation through the optimization step, challenging its applicability to a broad class of optimization problems. This paper proposes an alternative method, in which optimal solutions are learned directly from the observable features by joint predictive models. The approach is generic, and based on an adaptation of the Learning-to-Optimize paradigm, from which a rich variety of existing techniques can be employed. Experimental evaluations show the ability of several Learning-to-Optimize methods to provide efficient and accurate solutions to an array of challenging Predict-Then-Optimize problems.

Figures (7)

• Figure 1: Illustration of Learning to Optimize from Features, in relation to other learning paradigms.
• Figure 2: Effect of shifting inputs received by the LtO proxy: a mismatch from its initial training distribution leads to inaccurate solutions when employed in PtO training.
• Figure 3: Effect on regret as LtO proxy acts outside its training set.
• Figure 4: Comparison between LtO ($k\!=\!0$), LtOF, Two-stage (2S) and EPO ($k\!>\!1$) on the portfolio optimization. 2S(EPO)-$m$ indicates that the prediction model of the respective PtO method is an $m$ layer ReLU neural network. Plot y-axe is in semi log-scale.
• Figure 5: Comparison between LtO ($k=0$), LtOF, and Two Stage Method (2S) on the nonconvex QP (top) and AC-OPF case (bottom). Top plot y-axe is in semi log-scale.