Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CaVE: A Cone-Aligned Approach for Fast Predict-then-optimize with Binary Linear Programs

Bo Tang, Elias B. Khalil

TL;DR

This work focuses on binary linear programs (BLPs) and proposes a new end-to-end training method, Cone-aligned Vector Estimation (CaVE), that aligns the predicted cost vectors with the normal cone corresponding to the true optimal solution of a training instance.

Abstract

The end-to-end predict-then-optimize framework, also known as decision-focused learning, has gained popularity for its ability to integrate optimization into the training procedure of machine learning models that predict the unknown cost (objective function) coefficients of optimization problems from contextual instance information. Naturally, most of the problems of interest in this space can be cast as integer linear programs. In this work, we focus on binary linear programs (BLPs) and propose a new end-to-end training method to predict-then-optimize. Our method, Cone-aligned Vector Estimation (CaVE), aligns the predicted cost vectors with the normal cone corresponding to the true optimal solution of a training instance. When the predicted cost vector lies inside the cone, the optimal solution to the linear relaxation of the binary problem is optimal. This alignment not only produces decision-aware learning models but also dramatically reduces training time as it circumvents the need to solve BLPs to compute a loss function with its gradients. Experiments across multiple datasets show that our method exhibits a favorable trade-off between training time and solution quality, particularly with large-scale optimization problems such as vehicle routing, a hard BLP that has yet to benefit from predict-then-optimize methods in the literature due to its difficulty.

CaVE: A Cone-Aligned Approach for Fast Predict-then-optimize with Binary Linear Programs

TL;DR

This work focuses on binary linear programs (BLPs) and proposes a new end-to-end training method, Cone-aligned Vector Estimation (CaVE), that aligns the predicted cost vectors with the normal cone corresponding to the true optimal solution of a training instance.

Abstract

The end-to-end predict-then-optimize framework, also known as decision-focused learning, has gained popularity for its ability to integrate optimization into the training procedure of machine learning models that predict the unknown cost (objective function) coefficients of optimization problems from contextual instance information. Naturally, most of the problems of interest in this space can be cast as integer linear programs. In this work, we focus on binary linear programs (BLPs) and propose a new end-to-end training method to predict-then-optimize. Our method, Cone-aligned Vector Estimation (CaVE), aligns the predicted cost vectors with the normal cone corresponding to the true optimal solution of a training instance. When the predicted cost vector lies inside the cone, the optimal solution to the linear relaxation of the binary problem is optimal. This alignment not only produces decision-aware learning models but also dramatically reduces training time as it circumvents the need to solve BLPs to compute a loss function with its gradients. Experiments across multiple datasets show that our method exhibits a favorable trade-off between training time and solution quality, particularly with large-scale optimization problems such as vehicle routing, a hard BLP that has yet to benefit from predict-then-optimize methods in the literature due to its difficulty.
Paper Structure (28 sections, 12 equations, 4 figures, 15 tables, 1 algorithm)

This paper contains 28 sections, 12 equations, 4 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. KKT-Based Methods.
  4. Black-Box Methods.
  5. Function Approximation Methods.
  6. Problem Statement and Preliminaries
  7. Definitions and Notation
  8. Metric
  9. Methodology
  10. Optimal Cones and Subcones
  11. Cone-aligned Vector Estimation: three variants
  12. CaVE-E with exact projection.
  13. CaVE+ with inner projection.
  14. CaVE-H with a mix of inner and heuristic projections.
  15. Comparison with existing methods.
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Illustration of a normal cone: the cost vectors $\bm{c}_1$ and $\bm{c}_2$ produce the same optimal solution if and only if they lie within this cone.
  • Figure 2: Illustration of the optimal cone and optimal subcone: On the left, the green cone is the optimal cone of a BLP. On the right, the green cone is a subset of the left cone and the optimal cone of the LP relaxation of the BLP on the left.
  • Figure 3: Illustration of the three projections: Exact projection on the left, inner projection in the middle, and heuristic projection on the right.
  • Figure 4: Validation regeret curves for TSP50 with a polynomial of degree 4 (left) and 6 (right). The vertical axis represents the average normalized regret values of each of the five methods for the validation dataset with $1000$ instances.