Score Function Gradient Estimation to Widen the Applicability of Decision-Focused Learning

TL;DR

This work proposes an alternative method that combines stochastic smoothing with score function gradient estimation which works on any task loss and shows that it typically requires more epochs, but that it is on par with specialized methods and performs especially well for the difficult case of problems with uncertainty in the constraints.

Abstract

Many real-world optimization problems contain parameters that are unknown before deployment time, either due to stochasticity or to lack of information (e.g., demand or travel times in delivery problems). A common strategy in such cases is to estimate said parameters via machine learning (ML) models trained to minimize the prediction error, which however is not necessarily aligned with the downstream task-level error. The decision-focused learning (DFL) paradigm overcomes this limitation by training to directly minimize a task loss, e.g. regret. Since the latter has non-informative gradients for combinatorial problems, state-of-the-art DFL methods introduce surrogates and approximations that enable training. But these methods exploit specific assumptions about the problem structures (e.g., convex or linear problems, unknown parameters only in the objective function). We propose an alternative method that makes no such assumptions, it combines stochastic smoothing with score function gradient estimation which works on any task loss. This opens up the use of DFL methods to nonlinear objectives, uncertain parameters in the problem constraints, and even two-stage stochastic optimization. Experiments show that it typically requires more epochs, but that it is on par with specialized methods and performs especially well for the difficult case of problems with uncertainty in the constraints, in terms of solution quality, scalability, or both.

Figures (9)

• Figure 1: Illustration of a DFL loss with non-informative derivatives smoothed by predicting a Gaussian over the parameters with increasing variances. The larger the variance, the more the loss gets smoothed, but the less it resembles the original piecewise-constant task loss.
• Figure 2: The relative post-hoc regret and normalized runtime at inference time of SFGE and PFL+SAA on the WSMC of size $10\times50$, for a $\rho=5$ (left) and $\rho=10$ (right).
• Figure 3: Comparison between SFGE and PFL+SAA on the KP-50 with stochastic item weights, for $\rho=5$ (left) and $\rho=10$ (right).
• Figure 4: Left: validation regret on the KP-50 w.r.t. the number of epochs when multiple predictions $\hat{y}$ are sampled for the same $x$. Right: test relative regret on the KP-50 when $\sigma$ is contextual (predicted std dev) and a trainable parameter (trainable), compared with the state-of-the-art SPO.
• Figure 5: Validation relative regret during training of SFGE with and without standardization on the KP-50