Machine Learning Augmented Branch and Bound for Mixed Integer Linear Programming

TL;DR

This survey investigates how machine learning augments branch-and-bound solvers for mixed-integer linear programs, focusing on decision points such as primal heuristics, branching, cutting planes, node selection, and solver configuration. It surveys representations (notably graph-based MILP encodings) and learning paradigms (supervised and reinforcement learning) to map solver data to improved decisions, while emphasizing instance-specific generalization and the need for dynamic, data-driven solvers. Key contributions include a taxonomy of learning tasks, evaluation metrics, and coverage of datasets and software (e.g., MIPLIB, ML4CO, Ecole) that enable standardized benchmarking. The work highlights promising directions for adaptive, instance-aware solvers, while acknowledging challenges in generalization, hardware constraints, and integrating ML with traditional optimization software.

Abstract

Mixed Integer Linear Programming (MILP) is a pillar of mathematical optimization that offers a powerful modeling language for a wide range of applications. During the past decades, enormous algorithmic progress has been made in solving MILPs, and many commercial and academic software packages exist. Nevertheless, the availability of data, both from problem instances and from solvers, and the desire to solve new problems and larger (real-life) instances, trigger the need for continuing algorithmic development. MILP solvers use branch and bound as their main component. In recent years, there has been an explosive development in the use of machine learning algorithms for enhancing all main tasks involved in the branch-and-bound algorithm, such as primal heuristics, branching, cutting planes, node selection and solver configuration decisions. This paper presents a survey of such approaches, addressing the vision of integration of machine learning and mathematical optimization as complementary technologies, and how this integration can benefit MILP solving. In particular, we give detailed attention to machine learning algorithms that automatically optimize some metric of branch-and-bound efficiency. We also address how to represent MILPs in the context of applying learning algorithms, MILP benchmarks and software.

Figures (5)

• Figure 1: Markov Decision Process
• Figure 2: Embedding computation with a Graph Neural Network. Here, we use the abbreviation $\xi^t_v$ for $\xi^t(G,v)$. To update the embedding $\xi^t_v$ of node $v$ at time $t$, the embeddings of neighboring nodes are added and then combined with the current embedding using the comb function. The result is $\xi^{t+1}_v$.
• Figure 3: Three learning problems related to primal heuristics: (a) predict a reference solution and search in its neighborhood, (b) neighborhood selection -- which and/or how many variables to unfix and re-optimize, (c) heuristic scheduling -- which heuristics to run and/or for how long.
• Figure 4: Three models for learning to cut: (a) Tang2020Paulus2022, (b) Wang2023, (c) Turner2023. Here $PD(t_{max})$ refers to the primal-dual integral (see Section \ref{['subsec:metrics']}).
• Figure 5: The bipartite graph representation of an MILP.

Theorems & Definitions (3)

• Definition 1: Feed-forward Neural Network
• Definition 2: Graph embedding
• Definition 3: Graph Neural Network