Sufficient Decision Proxies for Decision-Focused Learning

TL;DR

This paper investigates when a single-scenario deterministic proxy in decision-focused learning (DFL) is sufficient and when it is not. It introduces sufficient decision proxies—scenario-based, quadratic, and parameterized-distribution approaches—that maintain low learning complexity while preserving decision quality, and analyzes both continuous and two-stage stochastic problems. Through experiments on portfolio optimization, weighted-set multi-cover, and probabilistic traveling salesman problems, the authors show that deterministic proxies can be suboptimal and that the proposed proxies frequently yield near-optimal decisions, even with small numbers of scenarios. The work provides theoretical conditions, practical alternatives, and empirical evidence that advance the reliability and applicability of DFL for real-world contextual optimization under uncertainty.

Abstract

When solving optimization problems under uncertainty with contextual data, utilizing machine learning to predict the uncertain parameters is a popular and effective approach. Decision-focused learning (DFL) aims at learning a predictive model such that decision quality, instead of prediction accuracy, is maximized. Common practice here is to predict a single value for each uncertain parameter, implicitly assuming that there exists a (single-scenario) deterministic problem approximation (proxy) that is sufficient to obtain an optimal decision. Other work assumes the opposite, where the underlying distribution needs to be estimated. However, little is known about when either choice is valid. This paper investigates for the first time problem properties that justify using either assumption. Using this, we present effective decision proxies for DFL, with very limited compromise on the complexity of the learning task. We show the effectiveness of presented approaches in experiments on problems with continuous and discrete variables, as well as uncertainty in the objective function and in the constraints.

Figures (4)

• Figure 1: Schematic representation of contextual optimization and the difference between the true optimal decision based on the unknown contextual distribution (orange) and the empirical optimal decision based on the deterministic proxy (blue)
• Figure 2: True objective function $\mathbb{E}_{c \sim \mathcal{U}[-1, 1.7]}f(c, x)$ compared to deterministic proxy objectives $f(\hat{c}, x)$ for Example \ref{['ex:kelly']}. $*$ denote different function maxima.
• Figure 3: Absolute regret mean on the test set, normalized by $\pi^\text{D}$. Error bars denote one standard deviation. $\pi^\text{D}_\text{PFL}$ bars for WSMC 1.80 (0.21) and PTSP 2.35 (0.35) were cut off for visibility.
• Figure 4: Validation learning curves per epoch ($x$-axis). The average of approaches minimum validation absolute regret was used to scale the absolute regret for each seed. Error bars denote one standard deviation.

Theorems & Definitions (21)

• Example 1
• Theorem 1
• Theorem 2
• Theorem 3
• Definition 1
• Lemma 1
• Theorem 4
• Definition 2
• Theorem 5
• Example 2