Fast and Interpretable Mixed-Integer Linear Program Solving by Learning Model Reduction

TL;DR

This work tackles the scalability and interpretability challenges of solving large MILPs by learning a reduced and equivalent MILP model as an intermediate step. It introduces a preference-based strategy learning framework that leverages performance information across many reduced models, augmented with an attention mechanism to capture instance–strategy relationships, and a SetCover-based pruning to control label growth. An online inference pipeline selects Top‑$k$ candidate reductions and picks the best by minimizing infeasibility, aided by potential KKT-based speedups. Empirical results on real-world MILP datasets show substantial speedups over Gurobi and improved solution quality compared to prior ML-based reduction methods, with strong feasibility and suboptimality performance and interpretable reduced strategies.

Abstract

By exploiting the correlation between the structure and the solution of Mixed-Integer Linear Programming (MILP), Machine Learning (ML) has become a promising method for solving large-scale MILP problems. Existing ML-based MILP solvers mainly focus on end-to-end solution learning, which suffers from the scalability issue due to the high dimensionality of the solution space. Instead of directly learning the optimal solution, this paper aims to learn a reduced and equivalent model of the original MILP as an intermediate step. The reduced model often corresponds to interpretable operations and is much simpler, enabling us to solve large-scale MILP problems much faster than existing commercial solvers. However, current approaches rely only on the optimal reduced model, overlooking the significant preference information of all reduced models. To address this issue, this paper proposes a preference-based model reduction learning method, which considers the relative performance (i.e., objective cost and constraint feasibility) of all reduced models on each MILP instance as preferences. We also introduce an attention mechanism to capture and represent preference information, which helps improve the performance of model reduction learning tasks. Moreover, we propose a SetCover based pruning method to control the number of reduced models (i.e., labels), thereby simplifying the learning process. Evaluation on real-world MILP problems shows that 1) compared to the state-of-the-art model reduction ML methods, our method obtains nearly 20% improvement on solution accuracy, and 2) compared to the commercial solver Gurobi, two to four orders of magnitude speedups are achieved.

Figures (7)

• Figure 1: Overview of our framework, which comprises two phases: 1) Strategy Generation and Pruning and 2) Preference-based Strategy Learning. In 1), the strategies are explored from instances and a SetCover is constructed to prune redundant strategies. In 2), an attention architecture is utilized to capture the ranked preference information over instances.
• Figure 2: The $k$ highest outputs of preference model are selected as candidate strategies. Given the strategy, the instance can be solved rapidly by model reduction.
• Figure 3: The performance on six scenarios from MIPLIB and each vertex in the subplot represents a metric. For better presentation of results (due to the differences in metric magnitudes), we map each metric to the range of (0, 100) using the same function for each metric. The Gurobi's result is shown on the left since its results are similarly presented across all scenarios
• Figure 4: Results on Fuel Cell Energy Management. (a) Performance under varying problem scale (larger scale as $T$ increasing), where $t_{\max}^H$ means the maximum computation time (in seconds) from Gurobi Heuristic and $t_{\max}^G$ is from Gurobi. (b) and (c) show the average infeasibility and suboptimality for our method (bar on the left) and MLOPT under $T$=60 and varying $k$.
• Figure 5: Performance on Inventory Management Problems