Managing Solution Stability in Decision-Focused Learning with Cost Regularization
Victor Spitzer, Francois Sanson
TL;DR
This work analyzes how perturbation-based differentiations in decision-focused learning for MILPs are governed by solution stability of the optimization mapping. It shows that misalignment between perturbation scale and cost-vector magnitude can cause learning to imitate existing solutions or fail to improve performance. The authors propose cost regularization strategies—norm-based projections that cap stability radii and preserve the same decision mapping—to stabilize training. Empirical results on benchmark problems and a targeted toy-study demonstrate that regularization, especially a projection-based approach, consistently improves regret and mitigates instability across seeds and datasets. The findings offer a principled, broadly applicable method to enhance robustness of DFL methods that rely on perturbations.
Abstract
Decision-focused learning integrates predictive modeling and combinatorial optimization by training models to directly improve decision quality rather than prediction accuracy alone. Differentiating through combinatorial optimization problems represents a central challenge, and recent approaches tackle this difficulty by introducing perturbation-based approximations. In this work, we focus on estimating the objective function coefficients of a combinatorial optimization problem. Our study demonstrates that fluctuations in perturbation intensity occurring during the learning phase can lead to ineffective training, by establishing a theoretical link to the notion of solution stability in combinatorial optimization. We propose addressing this issue by introducing a regularization of the estimated cost vectors which improves the robustness and reliability of the learning process, as demonstrated by extensive numerical experiments.