PySCIPOpt-ML: Embedding Trained Machine Learning Models into Mixed-Integer Programs

TL;DR

The paper tackles embedding trained ML predictors into MIPs when relationships are unknown or too complex for direct modeling. It presents PySCIPOpt-ML, an open-source tool that directly interfaces with multiple ML frameworks and the open-source solver SCIP to automatically formulate and embed predictors into MIPs, along with SurrogateLIB, a library of MIPs with embedded ML constraints. A key contribution is the development of practical, solver-friendly formulations, including SOS1-based ReLU representations and argmax encodings, as well as a broad treatment of tree-based predictors. Empirical results on SurrogateLIB illustrate the feasible scale of embedded predictors across problems and formulations, offering guidance on when and how to embed ML surrogates in practice and underscoring the approach's potential to accelerate decision-making with open-source tooling.

Abstract

A standard tool for modelling real-world optimisation problems is mixed-integer programming (MIP). However, for many of these problems, information about the relationships between variables is either incomplete or highly complex, making it difficult or even impossible to model the problem directly. To overcome these hurdles, machine learning (ML) predictors are often used to represent these relationships and are then embedded in the MIP as surrogate models. Due to the large amount of available ML frameworks and the complexity of many ML predictors, formulating such predictors into MIPs is a highly non-trivial task. In this paper, we introduce PySCIPOpt-ML, an open-source tool for the automatic formulation and embedding of trained ML predictors into MIPs. By directly interfacing with a broad range of commonly used ML frameworks and an open-source MIP solver, PySCIPOpt-ML provides a way to easily integrate ML constraints into optimisation problems. Alongside PySCIPOpt-ML, we introduce, SurrogateLIB, a library of MIP instances with embedded ML constraints, and present computational results over SurrogateLIB, providing intuition on the scale of ML predictors that can be practically embedded. The project is available at https://github.com/Opt-Mucca/PySCIPOpt-ML.

Figures (2)

• Figure 1: How trained ML predictors can be automatically formulated as MIPs and embedded in an optimisation problem using our Python package PySCIPOpt-ML.
• Figure 2: The workflow when embedding ML predictors into a MIP with PySCIPOpt-ML.