Differentiation of Blackbox Combinatorial Solvers

TL;DR

The paper presents a principled method to differentiate through blackbox combinatorial solvers with linear objectives by introducing a continuous interpolation, f_lambda, that yields informative gradients for training. It enables backpropagation through exact solvers such as Gurobi, Blossom V, and Dijkstra by solving a perturbed input in a single forward pass during the backward step. The authors provide theoretical justifications and validate the approach on three tasks—shortest path, globe TSP, and MNIST-based min-cost matching—demonstrating strong generalization and learning capabilities beyond standard neural nets. This fusion of neural networks with classical combinatorial algorithms offers a practical pathway to end-to-end learning in problems with large combinatorial structure and global constraints.

Abstract

Achieving fusion of deep learning with combinatorial algorithms promises transformative changes to artificial intelligence. One possible approach is to introduce combinatorial building blocks into neural networks. Such end-to-end architectures have the potential to tackle combinatorial problems on raw input data such as ensuring global consistency in multi-object tracking or route planning on maps in robotics. In this work, we present a method that implements an efficient backward pass through blackbox implementations of combinatorial solvers with linear objective functions. We provide both theoretical and experimental backing. In particular, we incorporate the Gurobi MIP solver, Blossom V algorithm, and Dijkstra's algorithm into architectures that extract suitable features from raw inputs for the traveling salesman problem, the min-cost perfect matching problem and the shortest path problem. The code is available at https://github.com/martius-lab/blackbox-backprop.

Figures (9)

• Figure 1: Architecture design enabled by Theorem \ref{['T:f-lambda']}. Blackbox combinatorial solver embedded into a neural network.
• Figure 2: Continuous interpolation of a piecewise constant function. (\ref{['fig:d-lambda-small']}) $f_\lambda$ for a small value of $\lambda$; the set $W^{\lambda}_{\text{eq}}$ is still substantial and only two interpolators $g_1$ and $g_2$ are incomplete. Also, all interpolators are $0$-interpolators. (\ref{['fig:d-lambda-big']}) $f_\lambda$ for a high value of $\lambda$; most interpolators are incomplete and we also encounter a $\delta$-interpolator $g_3$ (between $y_1$ and $y_2$) which attains the value $f(y_1)$$\delta$-away from the set $P_1$. Despite losing some local structure for high $\lambda$, the gradient of $f_\lambda$ is still informative.
• Figure 3: Example $f_\lambda$ for $w \in \mathbb{R}^2$ and $\lambda = 3, 10, 20$ (left to right). As $\lambda$ changes, the interpolation $f_\lambda$ is less faithful to the piecewise constant $f(y(w))$ but provides reasonable gradient on a larger set.
• Figure 4: The SP$(k)$ dataset. (\ref{['fig:sp-results:dataset']}) Each input is a $k \times k$ grid of tiles corresponding to a Warcraft II terrain map, the respective label is a the matrix indicating the shortest path from top left to bottom right. (\ref{['fig:sp-results:withpath']}) is a different map with correctly predicted shortest path.
• Figure 5: The TSP($k$) problem. (\ref{['fig:tsp:dataset']}) illustrates the dataset. Each input is a sequence of $k$ flags and the corresponding label is the adjacency matrix of the optimal TSP tour around the corresponding capitals. (\ref{['fig:tsp:oceania']}) displays the learned locations of 10 country capitals in southeast Asia and Australia, accurately recovering their true position.

Theorems & Definitions (5)

• Example 1: Encoding shortest-path problem
• Theorem 1
• Remark 1
• Proposition 1
• Remark 2