Sample Complexity of Algorithm Selection Using Neural Networks and Its Applications to Branch-and-Cut
Hongyu Cheng, Sammy Khalife, Barbara Fiedorowicz, Amitabh Basu
TL;DR
This paper advances data-driven algorithm design by showing how neural networks can learn instance-dependent mappings from problem instances to cutting-plane decisions in branch-and-cut. By framing the mapping as a parameterized NN problem and bounding the pseudo-dimension of the resulting hypothesis class, it provides rigorous sample complexity guarantees for neural-network-based algorithm selection. The authors apply this framework to Chvátal-Gomory and Gomory mixed-integer cuts at the root node of MILP branch-and-cut trees, deriving both single-cut and multi-cut learnability bounds under Sigmoid and ReLU activations. Empirically, they demonstrate that neural-network-guided cut selection yields significant reductions in tree size and faster cut selection compared with traditional, tableaus-based methods, using reinforcement learning to train the model on knapsack instances.
Abstract
Data-driven algorithm design is a paradigm that uses statistical and machine learning techniques to select from a class of algorithms for a computational problem an algorithm that has the best expected performance with respect to some (unknown) distribution on the instances of the problem. We build upon recent work in this line of research by considering the setup where, instead of selecting a single algorithm that has the best performance, we allow the possibility of selecting an algorithm based on the instance to be solved, using neural networks. In particular, given a representative sample of instances, we learn a neural network that maps an instance of the problem to the most appropriate algorithm for that instance. We formalize this idea and derive rigorous sample complexity bounds for this learning problem, in the spirit of recent work in data-driven algorithm design. We then apply this approach to the problem of making good decisions in the branch-and-cut framework for mixed-integer optimization (e.g., which cut to add?). In other words, the neural network will take as input a mixed-integer optimization instance and output a decision that will result in a small branch-and-cut tree for that instance. Our computational results provide evidence that our particular way of using neural networks for cut selection can make a significant impact in reducing branch-and-cut tree sizes, compared to previous data-driven approaches.