Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learning

Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra Dilkina, Benoit Steiner

TL;DR

The paper tackles solving large, hard ILPs by improving Large Neighborhood Search through learning a destroy policy. It introduces CL-LNS, which uses a contrastive loss to train a Graph Attention Network-based policy that imitatesLB-derived good subsets while leveraging intermediate LB solutions and perturbations to form positive and negative samples. Empirical results across MVC, MIS, CA, and SC demonstrate state-of-the-art anytime performance, strong generalization to larger instances, and clear gains from the contrastive objective and richer features. The approach offers practical impact by delivering faster, higher-quality primal solutions and providing a framework applicable to a broad class of COPs that require efficient neighborhood selection. It also opens avenues for integrating learned destroy policies with BnB and extending contrastive learning to other combinatorial search subproblems.

Abstract

Integer Linear Programs (ILPs) are powerful tools for modeling and solving a large number of combinatorial optimization problems. Recently, it has been shown that Large Neighborhood Search (LNS), as a heuristic algorithm, can find high quality solutions to ILPs faster than Branch and Bound. However, how to find the right heuristics to maximize the performance of LNS remains an open problem. In this paper, we propose a novel approach, CL-LNS, that delivers state-of-the-art anytime performance on several ILP benchmarks measured by metrics including the primal gap, the primal integral, survival rates and the best performing rate. Specifically, CL-LNS collects positive and negative solution samples from an expert heuristic that is slow to compute and learns a new one with a contrastive loss. We use graph attention networks and a richer set of features to further improve its performance.

Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learning

TL;DR

The paper tackles solving large, hard ILPs by improving Large Neighborhood Search through learning a destroy policy. It introduces CL-LNS, which uses a contrastive loss to train a Graph Attention Network-based policy that imitatesLB-derived good subsets while leveraging intermediate LB solutions and perturbations to form positive and negative samples. Empirical results across MVC, MIS, CA, and SC demonstrate state-of-the-art anytime performance, strong generalization to larger instances, and clear gains from the contrastive objective and richer features. The approach offers practical impact by delivering faster, higher-quality primal solutions and providing a framework applicable to a broad class of COPs that require efficient neighborhood selection. It also opens avenues for integrating learned destroy policies with BnB and extending contrastive learning to other combinatorial search subproblems.

Abstract

Integer Linear Programs (ILPs) are powerful tools for modeling and solving a large number of combinatorial optimization problems. Recently, it has been shown that Large Neighborhood Search (LNS), as a heuristic algorithm, can find high quality solutions to ILPs faster than Branch and Bound. However, how to find the right heuristics to maximize the performance of LNS remains an open problem. In this paper, we propose a novel approach, CL-LNS, that delivers state-of-the-art anytime performance on several ILP benchmarks measured by metrics including the primal gap, the primal integral, survival rates and the best performing rate. Specifically, CL-LNS collects positive and negative solution samples from an expert heuristic that is slow to compute and learns a new one with a contrastive loss. We use graph attention networks and a richer set of features to further improve its performance.
Paper Structure (38 sections, 10 equations, 10 figures, 12 tables)

This paper contains 38 sections, 10 equations, 10 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Background
  3. ILPs
  4. LNS for ILP solving
  5. Adaptive Neighborhood Size
  6. LB Heuristic
  7. Related Work
  8. LNS for ILPs and Other COPs
  9. Learning to Solve ILPs with BnB
  10. Contrastive Learning for COPs
  11. Contrastive Learning for LNS
  12. Data Collection
  13. Collecting Positive Samples $\mathcal{S}_{\mathsf{p}}^t$
  14. Collecting Negative Samples $\mathcal{S}_{\mathsf{n}}^t$
  15. Policy Network
  16. ...and 23 more sections

Figures (10)

  • Figure 1: The primal gap (the lower the better) as a function of runtime, averaged over 100 test instances. For ML approaches, the policies are trained on only small training instances but tested on both small and large test instances.
  • Figure 2: The survival rate (the higher the better) over 100 test instances as a function of runtime to meet primal gap threshold 1.00%. For ML approaches, the policies are trained on only small training instances but tested on both small and large test instances.
  • Figure 3: The best performing rate (the higher the better) as a function of runtime on 100 small instances (see Appendix for results on large instances). The sum of the best performing rates at a given runtime might sum up greater than 1 since ties are counted multiple times.
  • Figure 4: The primal bound (the lower the better) as a function of number of iterations, averaged over 100 small test instances. LB and LB (data collection) are LNS with LB using the neighborhood sizes fune-tunded for CL-LNS and for data collection, respectively. The table shows the neighborhood size (NH size) and the average runtime in seconds (with standard deviations) per iteration for each approach.
  • Figure 5: Ablation study: The primal gap (the lower the better) as a function of time, averaged over 100 small test instances.
  • ...and 5 more figures