Learning Branching Policies for MILPs with Proximal Policy Optimization

TL;DR

The paper tackles the challenge of learning branching policies for MILP solvers by introducing Tree-Gate PPO (TGPPO), an on-policy reinforcement learning framework that trains a branching policy directly through interactions with a Branch-and-Bound solver. It combines a permutation-equivariant Tree-Gate Transformer with an actor-critic PPO objective to handle variable candidate sets and evolving tree context, while employing instance-normalized rewards to stabilize training. Through a two-phase training pipeline and nested cross-validation, TGPPO demonstrates improved efficiency, reducing the number of explored nodes on easy instances and achieving lower primal-dual integrals on hard instances, often outperforming prior learning-based branchers and approaching traditional strong heuristics. The results suggest that reinforcement learning, when paired with tree-aware architectures and robust training signals, can yield generalizable and scalable branching strategies for heterogeneous MILP instances, with strong practical implications for solver performance.

Abstract

Branch-and-Bound (B\&B) is the dominant exact solution method for Mixed Integer Linear Programs (MILP), yet its exponential time complexity poses significant challenges for large-scale instances. The growing capabilities of machine learning have spurred efforts to improve B\&B by learning data-driven branching policies. However, most existing approaches rely on Imitation Learning (IL), which tends to overfit to expert demonstrations and struggles to generalize to structurally diverse or unseen instances. In this work, we propose Tree-Gate Proximal Policy Optimization (TGPPO), a novel framework that employs Proximal Policy Optimization (PPO), a Reinforcement Learning (RL) algorithm, to train a branching policy aimed at improving generalization across heterogeneous MILP instances. Our approach builds on a parameterized state space representation that dynamically captures the evolving context of the search tree. Empirical evaluations show that TGPPO often outperforms existing learning-based policies in terms of reducing the number of nodes explored and improving p-Primal-Dual Integrals (PDI), particularly in out-of-distribution instances. These results highlight the potential of RL to develop robust and adaptable branching strategies for MILP solvers.

Figures (2)

• Figure 1: The Tree–Gate PPO agent interacts with SCIP through an environment wrapper: candidate features and local tree features are encoded, the actor chooses a branching variable, SCIP executes the branch, and the environment returns rewards. Trajectories accumulate in a buffer and are periodically used for PPO optimisation, closing the learning loop.
• Figure 2: Head-to-head scatter of tgppo vs. tbrant (log–log): points below the dashed diagonal ($y<x$) indicate tgppo wins; (a) comparison of the easy instances using Nnodes, (b) comparison of the hard instances using PDI