Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Backdoors for Mixed Integer Linear Programs with Contrastive Learning

Junyang Cai, Taoan Huang, Bistra Dilkina

TL;DR

The paper addresses speeding up MILP solving by predicting effective backdoors—small subsets of binary variables to prioritize in branching. It introduces a contrastive-learning framework that uses Monte-Carlo Tree Search (MCTS) to collect high-quality backdoors and trains a Graph Attention Network with an InfoNCE loss to predict backdoors from a bipartite MILP graph representation. Empirical results across GISP, SC, CA, MIS, FC, and NN demonstrate that the proposed method achieves faster solve times than Gurobi and prior learning-based approaches, with strong generalization to larger instances. The approach reduces training randomness and offers a deterministic, scalable way to apply ML to MILP backdoor prediction, though it relies on offline data collection per problem distribution and invites future theoretical exploration of backdoors and broader CO applications.

Abstract

Many real-world problems can be efficiently modeled as Mixed Integer Linear Programs (MILPs) and solved with the Branch-and-Bound method. Prior work has shown the existence of MILP backdoors, small sets of variables such that prioritizing branching on them when possible leads to faster running times. However, finding high-quality backdoors that improve running times remains an open question. Previous work learns to estimate the relative solver speed of randomly sampled backdoors through ranking and then decide whether to use the highest-ranked backdoor candidate. In this paper, we utilize the Monte-Carlo tree search method to collect backdoors for training, rather than relying on random sampling, and adapt a contrastive learning framework to train a Graph Attention Network model to predict backdoors. Our method, evaluated on several common MILP problem domains, demonstrates performance improvements over both Gurobi and previous models.

Learning Backdoors for Mixed Integer Linear Programs with Contrastive Learning

TL;DR

The paper addresses speeding up MILP solving by predicting effective backdoors—small subsets of binary variables to prioritize in branching. It introduces a contrastive-learning framework that uses Monte-Carlo Tree Search (MCTS) to collect high-quality backdoors and trains a Graph Attention Network with an InfoNCE loss to predict backdoors from a bipartite MILP graph representation. Empirical results across GISP, SC, CA, MIS, FC, and NN demonstrate that the proposed method achieves faster solve times than Gurobi and prior learning-based approaches, with strong generalization to larger instances. The approach reduces training randomness and offers a deterministic, scalable way to apply ML to MILP backdoor prediction, though it relies on offline data collection per problem distribution and invites future theoretical exploration of backdoors and broader CO applications.

Abstract

Many real-world problems can be efficiently modeled as Mixed Integer Linear Programs (MILPs) and solved with the Branch-and-Bound method. Prior work has shown the existence of MILP backdoors, small sets of variables such that prioritizing branching on them when possible leads to faster running times. However, finding high-quality backdoors that improve running times remains an open question. Previous work learns to estimate the relative solver speed of randomly sampled backdoors through ranking and then decide whether to use the highest-ranked backdoor candidate. In this paper, we utilize the Monte-Carlo tree search method to collect backdoors for training, rather than relying on random sampling, and adapt a contrastive learning framework to train a Graph Attention Network model to predict backdoors. Our method, evaluated on several common MILP problem domains, demonstrates performance improvements over both Gurobi and previous models.
Paper Structure (26 sections, 12 equations, 5 figures, 6 tables)

This paper contains 26 sections, 12 equations, 5 figures, 6 tables.

Table of Contents

  1. Background
  2. Mixed Integer Linear Programming
  3. Backdoors for MILP
  4. Related Work
  5. Backdoors For Combinatorial Problems
  6. Machine Learning for MILPs
  7. Contrastive Learning for CO
  8. Contrastive Learning for Backdoor Prediction
  9. Data Collection
  10. Data representation and Policy network
  11. Training with Contrastive Loss and Applying Learned Policy
  12. Empirical Evaluation
  13. Problem Domains and Instance Generation
  14. Experiment Setup
  15. Results
  16. ...and 11 more sections

Figures (5)

  • Figure 1: This figure illustrates the comprehensive training and evaluation phases for learning backdoors for MILPs with contrastive learning. In the training phase, we employ MCTS to gather backdoor samples on training instances and collect positive and negative samples. Simultaneously, a feature extractor transforms the MILP instances into a bipartite graph and collects features of the MILPs. The policy is represented by a graph attention network (GAT) and trained with a contrastive loss. In the evaluation phase, the MILP instance is converted into a bipartite graph as input for the GAT, generating a score vector as output. We greedily select variables with the highest score to obtain backdoors, which are solved using a MILP solver.
  • Figure 2: Histograms illustrating the normalized runtime distributions of candidate backdoors for six distinct problem domains: (a) GISP-S, (b) SC-S, (c) CA-S, (d) IS-S, (e) FC, and (f) NN. The normalized runtime is calculated as the ratio of the solve time with the candidate backdoor to the original solve time without it. The red vertical line at 1.0 marks the threshold where the candidate backdoor's performance equals the original solve time. Values to the left of this line indicate instances where the candidate backdoor resulted in a faster solve time, while values to the right indicate a slower solve time than the original.
  • Figure 3: This figure shows the runtime of Gurobi (grb) and contrastive learning model (cl) on GISP-L, SC-L, CA-L, and MIS-L through two different types of plots. The left part is a scatter plot with Gurobi runtime as the x-axis and the speed improvement as the percentage of cl over grb as the y-axis. The points above the red line are ones where cl is better than grb and vice versa. The right part shows the finish rate as a function of runtime. The finish rate for a given runtime is the fraction of instances solved to optimality within the runtime. Every dot on the lines indicates one finished instance. The figures show that cl outperforms grb on average and specifically provides speedups on the harder instances in each distribution.
  • Figure 4: Histograms illustrating the normalized runtime distributions of best candidate backdoors among each instance for six distinct problem domains: (a) GISP-S, (b) SC-S, (c) CA-S, (d) IS-S, (e) FC, and (f) NN. The normalized runtime is calculated as the ratio of the solve time with the candidate backdoor to the original solve time without it. The red vertical line at 1.0 marks the threshold where the candidate backdoor's performance equals the original solve time. Values to the left of this line indicate instances where the candidate backdoor resulted in a faster solve time, while values to the right indicate a slower solve time than the original.
  • Figure 5: This figure shows the runtime of Gurobi (grb) and contrastive learning model (cl) on GISP-S, SC-S, CA-S, and MIS-S through two different types of plots. The left part is a scatter plot with Gurobi runtime as the x-axis and the speed improvement as the percentage of cl over grb as the y-axis. The points above the red line are ones where cl is better than grb and vice versa. The right part shows the finish rate as a function of runtime. The finish rate for a given runtime is the fraction of instances solved to optimality within the runtime. The blue line is grb and the orange line is cl, with every dot indicating one finished instance. The figures show that cl outperforms grb on average and specifically provides speedups on the harder instances in each distribution.