Learning-Based Hierarchical Approach for Fast Mixed-Integer Optimization

TL;DR

This work introduces a learning-based hierarchical approach to accelerate structured MIPs by decomposing a large problem into an upper-level and a lower-level optimization, with the upper-level decisions serving as parameters for the lower-level constraints. The authors formulate a convex, decision-focused training objective for predicting the upper-level cost vector, and develop multiple differentiable convex surrogates (including GSPO+, ASL, Z, and Fenchel-Young variants) to enable tractable learning of the optimizer. A key robustness component uses conformal prediction to provide probabilistic bounds on suboptimality, calibrated on separate data, offering online guarantees for the quality of predicted solutions. Through experiments on hierarchical knapsack, capacitated facility location, and multi-agent routing problems, the method achieves substantial speedups over state-of-the-art solvers while maintaining feasible and high-quality solutions, with conformal bounds offering reliable risk control. Together, these contributions advance practical, scalable optimization in data-rich settings where rapid reoptimization is essential.

Abstract

We propose a hierarchical architecture for efficiently computing high-quality solutions to structured mixed-integer programs (MIPs). To reduce computational effort, our approach decouples the original problem into a higher level problem and a lower level problem, both of smaller size. We solve both problems sequentially, where decisions of the higher level problem become parameters of the constraints of the lower level problem. We formulate this learning task as a convex optimization problem using decision-focused learning techniques and solve it by differentiating through the higher and the lower level problems in our architecture. To ensure robustness, we derive out-of-sample performance guarantees using conformal prediction. Numerical experiments in facility location, knapsack problems, and vehicle routing problems demonstrate that our approach significantly reduces computation time while maintaining feasibility and high solution quality compared to state-of-the-art solvers.

Figures (7)

• Figure 1: Top: Single, large optimization problem. Exact but may be intractible. Bottom: Hierarchical integer programming problems to approximate the top problem. Inexact but fast, with learnable parameters.
• Figure 2: The layered optimization model setup used to compute feasible solutions and upper-bounds.
• Figure 3: The full architecture of our model to predict a feasible solution to the top problem $\hat{x}$ and obtain a bound for its suboptimality. Here we write $\textbf{rel}(X)$ to mean the convex set given by relaxing integrality constraints in the set $X$.
• Figure 4: Results for the knapsack experiment. The left plot shows evaluation regret throughout the training process. The right plot shows the test regret \ref{['eq:absgap']} and time (averaged over the test instances) on the test set. The first heuristic to terimnate for Gurobi and SCIP terminates after finding a single feasible solution. The second to terminate terminates after finding three.
• Figure 5: Results for the facility location experiment. The left plot shows evaluation regret throughout the training process. The right plot shows the test regret \ref{['eq:absgap']} and time (averaged over the test instances) on the test set. The first heuristic to terimnate for Gurobi and SCIP terminates after finding a single feasible solution. The second to terminate terminates after finding three.

Theorems & Definitions (10)

• Theorem 3.1
• Lemma 3.1
• proof
• proof
• Theorem 3.2
• proof
• Theorem 4.1
• Lemma 4.1
• proof
• proof