Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph-Based Imitation and Reinforcement Learning for Efficient Benders Decomposition

Bernard T. Agyeman, Zhe Li, Ilias Mitrai, Prodromos Daoutidis

TL;DR

This work introduces an end-to-end graph-based agent for accelerating the computational efficiency of Benders Decomposition that is trained using a two-stage approach that combines imitation (IL) and reinforcement learning (RL).

Abstract

This work introduces an end-to-end graph-based agent for accelerating the computational efficiency of Benders Decomposition. The agent's policy is parameterized by a graph neural network which takes as input a bipartite graph representation of the master problem and proposes a candidate solution. The agent is trained using a two-stage approach that combines imitation (IL) and reinforcement learning (RL). IL is used to mimic a solver and obtain a warm-start policy which is then finetuned using RL with a reward signal that balances feasibility and computational efficiency. We augment the agent with a verification mechanism that checks the agent's prediction for feasibility and solution quality. The framework is evaluated in two case studies: (i) an illustrative mixed-integer nonlinear program, where it reduces the solution time by 42% without loss of solution quality, and (ii) a closed-loop irrigation scheduling problem, where it achieves a 23% time reduction without compromising water use efficiency.

Graph-Based Imitation and Reinforcement Learning for Efficient Benders Decomposition

TL;DR

This work introduces an end-to-end graph-based agent for accelerating the computational efficiency of Benders Decomposition that is trained using a two-stage approach that combines imitation (IL) and reinforcement learning (RL).

Abstract

This work introduces an end-to-end graph-based agent for accelerating the computational efficiency of Benders Decomposition. The agent's policy is parameterized by a graph neural network which takes as input a bipartite graph representation of the master problem and proposes a candidate solution. The agent is trained using a two-stage approach that combines imitation (IL) and reinforcement learning (RL). IL is used to mimic a solver and obtain a warm-start policy which is then finetuned using RL with a reward signal that balances feasibility and computational efficiency. We augment the agent with a verification mechanism that checks the agent's prediction for feasibility and solution quality. The framework is evaluated in two case studies: (i) an illustrative mixed-integer nonlinear program, where it reduces the solution time by 42% without loss of solution quality, and (ii) a closed-loop irrigation scheduling problem, where it achieves a 23% time reduction without compromising water use efficiency.

Paper Structure

This paper contains 40 sections, 18 equations, 6 figures, 4 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Formulation
  4. Generalized Benders Decomposition (GBD)
  5. Graph Representation of Optimization Problems
  6. Graph Neural Networks
  7. Reinforcement Learning (RL)
  8. Imitation Learning (IL)
  9. Graph-Based Agent
  10. Bipartite Graph Representation of the Master Problem
  11. IL Stage
  12. RL Stage
  13. State
  14. Action
  15. Transition Dynamics
  16. ...and 25 more sections

Figures (6)

  • Figure 1: Diagram of the proposed framework.
  • Figure 2: Bipartite graph of the master problem. The constraints are linear because the pure binary constraints are linear and the binary variables enter the objective and constraints of the original MINLP linearly.
  • Figure 3: A schematic of the GNN with multi-headed sigmoid output layer.
  • Figure 4: Percentage of confident and free binary variable assignments produced by each agent-based method over 100 test instances.
  • Figure 5: Evolution of the median Benders gap across 20 problem instances, plotted over the median number of iterations for each solution method.
  • ...and 1 more figures

Theorems & Definitions (3)

  • Remark 1
  • Remark 2
  • Remark 3