Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predicting Accurate Lagrangian Multipliers for Mixed Integer Linear Programs

Francesco Demelas, Joseph Le Roux, Mathieu Lacroix, Axel Parmentier

TL;DR

This work introduces a deep learning approach that bypasses the descent, effectively amortizing the local, per instance, optimization of Lagrangian relaxation, and provides a high quality warm-start for descent based Lagrangia methods.

Abstract

Lagrangian relaxation stands among the most efficient approaches for solving a Mixed Integer Linear Programs (MILP) with difficult constraints. Given any duals for these constraints, called Lagrangian Multipliers (LMs), it returns a bound on the optimal value of the MILP, and Lagrangian methods seek the LMs giving the best such bound. But these methods generally rely on iterative algorithms resembling gradient descent to maximize the concave piecewise linear dual function: the computational burden grows quickly with the number of relaxed constraints. We introduce a deep learning approach that bypasses the descent, effectively amortizing the local, per instance, optimization. A probabilistic encoder based on a graph convolutional network computes high-dimensional representations of relaxed constraints in MILP instances. A decoder then turns these representations into LMs. We train the encoder and decoder jointly by directly optimizing the bound obtained from the predicted multipliers. Numerical experiments show that our approach closes up to 85~\% of the gap between the continuous relaxation and the best Lagrangian bound, and provides a high quality warm-start for descent based Lagrangian methods.

Predicting Accurate Lagrangian Multipliers for Mixed Integer Linear Programs

TL;DR

This work introduces a deep learning approach that bypasses the descent, effectively amortizing the local, per instance, optimization of Lagrangian relaxation, and provides a high quality warm-start for descent based Lagrangia methods.

Abstract

Lagrangian relaxation stands among the most efficient approaches for solving a Mixed Integer Linear Programs (MILP) with difficult constraints. Given any duals for these constraints, called Lagrangian Multipliers (LMs), it returns a bound on the optimal value of the MILP, and Lagrangian methods seek the LMs giving the best such bound. But these methods generally rely on iterative algorithms resembling gradient descent to maximize the concave piecewise linear dual function: the computational burden grows quickly with the number of relaxed constraints. We introduce a deep learning approach that bypasses the descent, effectively amortizing the local, per instance, optimization. A probabilistic encoder based on a graph convolutional network computes high-dimensional representations of relaxed constraints in MILP instances. A decoder then turns these representations into LMs. We train the encoder and decoder jointly by directly optimizing the bound obtained from the predicted multipliers. Numerical experiments show that our approach closes up to 85~\% of the gap between the continuous relaxation and the best Lagrangian bound, and provides a high quality warm-start for descent based Lagrangian methods.
Paper Structure (39 sections, 16 equations, 2 figures, 7 tables)

This paper contains 39 sections, 16 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Learning Framework
  3. Overall Architecture
  4. Objective
  5. Encoding and Decoding Instances
  6. Encoder
  7. Decoder
  8. Link with Energy Based Models in structured prediction
  9. Related Work
  10. Evaluation
  11. Problems and Datasets
  12. Multi-Commodity Fixed-Charge Network Design (MC)
  13. Generalized Assignment (GA)
  14. Numerical Results
  15. Metrics
  16. ...and 24 more sections

Figures (2)

  • Figure 1: Overall Architecture. From the bipartite graph representation of a MILP and its CR solution, the model computes a Lagrangian dual solution. First the MILP is encoded by a GNN, from which we parameterize a sampler for constraint representations. These representations are then passed through a decoder to compute Lagrangian Multipliers.
  • Figure 2: The Graph Neural Network block. The first part is graph message-passing: we apply layer normalization to node features, then convolution over the instance's bipartite graph representation and finally dropout. The second phase consists of normalization, a Multi-Layer perceptron in parallel over all the nodes of the bipartite graph, then dropout. Both sublayers use residual connection between input and output. We apply this block several times to improve feature representations.