Machine Learning Insides OptVerse AI Solver: Design Principles and Applications

TL;DR

The paper addresses the scarcity of real-world mathematical programming instances and the need for faster, more accurate solvers by integrating ML into Huawei's OptVerse AI Solver. It introduces data-generation pipelines (HardSATGEN for SAT, G2MILP for MILP) and an AdaSolver augmentation framework to improve solver robustness, plus policy-learning components (GCN-based initial basis, RL4Presolve, HEM for cut selection, Neural Diving) and a comprehensive hyperparameter-tuning framework (HEBO, Transformer BO, DE variants). Empirical results on real-world Huawei benchmarks and standard MILP libraries demonstrate substantial improvements in speed and solution quality over traditional solvers like Gurobi, CPLEX, and SCIP, underscoring the practical impact of ML-driven strategies in combinatorial optimization. The work outlines a cohesive architecture spanning data, policy, and tuning layers, and points toward future directions including integration with large language models for problem formulation and solver interaction. Overall, the proposed ML-augmented approach broadens the applicability and effectiveness of mathematical programming solvers in industrial settings.

Abstract

In an era of digital ubiquity, efficient resource management and decision-making are paramount across numerous industries. To this end, we present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI Solver, which aims to mitigate the scarcity of real-world mathematical programming instances, and to surpass the capabilities of traditional optimization techniques. We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem. Furthermore, we introduce a training framework leveraging augmentation policies to maintain solvers' utility in dynamic environments. Besides the data generation and augmentation, our proposed approaches also include novel ML-driven policies for personalized solver strategies, with an emphasis on applications like graph convolutional networks for initial basis selection and reinforcement learning for advanced presolving and cut selection. Additionally, we detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance. Compared with traditional solvers such as Cplex and SCIP, our ML-augmented OptVerse AI Solver demonstrates superior speed and precision across both established benchmarks and real-world scenarios, reinforcing the practical imperative and effectiveness of machine learning techniques in mathematical programming solvers.

Figures (11)

• Figure 1: The trend of utilizing machine learning techniques to directly solve or to aid in solving the combinatorial problems in recent years. Several seminal works are listed here, especially for data generation, policy learning and hyper-parameter tuning techniques for mathematical programming solvers. Since 2023, this field draws more attentions than ever.
• Figure 2: The global picture of integrating machine learning techniques into OptVerse AI Solver. There are three main layers of integration, namely, data generation and augmentation for solvers (data layer), policy learning within solvers (policy layer), and parameter tuning for solvers (tuning layer).
• Figure 3: Overview of the HardSATGEN pipeline for SAT instance generation.
• Figure 4: We investigate two distinct task settings for MILP instance generation: (left) realistic MILP instance generation and (right) hard MILP instance generation.
• Figure 5: Overview of the G2MILP pipeline for MILP instance generation.