Combinatorial Optimization Augmented Machine Learning

TL;DR

Combinatorial Optimization Augmented Machine Learning (COAML) fuses predictive models with combinatorial decision-making by embedding optimization oracles as differentiable layers in end-to-end learning pipelines. The paper offers a unified formalization, a running taxonomy of problem settings (static/dynamic, explicit/implicit uncertainty), and a comprehensive methodology comprising architectural families (independent scoring, GNNs, autoregressive, domain-informed encoders) and learning strategies (empirical cost minimization, imitation learning, reinforcement learning). It surveys applications across hard optimization, two-stage and multi-stage stochastic problems, and data-driven perception-to-optimization tasks, distilling current state-of-the-art methods and identifying gaps (gradient through non-diff layers, scalability, distributional shift, benchmarks). By presenting design guidelines, theoretical insights, and a forward-looking research agenda, the work positions COAML as a practical and theoretically grounded framework for decision-focused learning at scale. The practical impact lies in enabling scalable, feasible, and data-driven decision policies for complex industrial and scientific problems where traditional predict-then-optimize pipelines underperform due to misalignment with downstream costs.

Abstract

Combinatorial optimization augmented machine learning (COAML) has recently emerged as a powerful paradigm for integrating predictive models with combinatorial decision-making. By embedding combinatorial optimization oracles into learning pipelines, COAML enables the construction of policies that are both data-driven and feasibility-preserving, bridging the traditions of machine learning, operations research, and stochastic optimization. This paper provides a comprehensive overview of the state of the art in COAML. We introduce a unifying framework for COAML pipelines, describe their methodological building blocks, and formalize their connection to empirical cost minimization. We then develop a taxonomy of problem settings based on the form of uncertainty and decision structure. Using this taxonomy, we review algorithmic approaches for static and dynamic problems, survey applications across domains such as scheduling, vehicle routing, stochastic programming, and reinforcement learning, and synthesize methodological contributions in terms of empirical cost minimization, imitation learning, and reinforcement learning. Finally, we identify key research frontiers. This survey aims to serve both as a tutorial introduction to the field and as a roadmap for future research at the interface of combinatorial optimization and machine learning.

Figures (5)

• Figure 1: Conceptual illustration of the paradigm. A parameterizes a surrogate problem. We propagate gradients backwards through the layer, enabling end-to-end training of the policy $\pi_w$ to minimize a downstream decision loss.
• Figure 2: Neural network with a -layer for the stochastic vehicle scheduling problem. Vertices represent requests. Dotted arrows represent arcs $a$, which are pairs of requests that can be operated in a sequence. The colored paths give vehicle routes in the solution returned. Image adapted from dalle2022learning.
• Figure 4: Schematic pipeline. An input instance $x$ is encoded by a statistical model $\varphi_w$, which parametrizes a oracle $\hat{y}$ to produce a decision $\pi_w(x)$. Different encoder architectures can be used depending on the problem class: independent scoring models, , sequential/constructive models, or domain-informed encoders.
• Figure 5: Normal cone to $\bar{y}$ and normal fan of $\mathcal{C}$. Oracle $\hat{y}$ is piecewise constant on each cone of the normal fan.
• Figure 6: Overview of the components of .

Theorems & Definitions (4)

• Example 1
• Example 2
• Example 3
• Example 4