Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Symbolic Optimization for Combinatorial Optimization: Accelerating Node Selection by Discovering Potential Heuristics

Hongyu Liu, Haoyang Liu, Yufei Kuang, Jie Wang, Bin Li

TL;DR

This work addresses the challenge of efficiently solving NP-hard combinatorial optimization problems by learning node-selection heuristics for Branch-and-Bound solvers. It introduces Dso4NS, a deep symbolic optimization framework that uses a sequential symbolic-regression model to generate compact, human-readable expressions for node scoring, trained via reinforcement learning with a behavioral-cloning-like fitness signal. Experiments on three benchmark families demonstrate that CPU-based symbolic policies can outperform traditional baselines and rival GPU-based methods in solving time, node exploration, and PD integral, while preserving interpretability and fast inference. By marrying the strengths of handcrafted heuristics and data-driven learning, Dso4NS offers a practical, interpretable pathway to enhance solver efficiency without relying on heavy neural networks during deployment.

Abstract

Combinatorial optimization (CO) is one of the most fundamental mathematical models in real-world applications. Traditional CO solvers, such as Branch-and-Bound (B&B) solvers, heavily rely on expert-designed heuristics, which are reliable but require substantial manual tuning. Recent studies have leveraged deep learning (DL) models as an alternative to capture rich feature patterns for improved performance on GPU machines. Nonetheless, the drawbacks of high training and inference costs, as well as limited interpretability, severely hinder the adoption of DL methods in real-world applications. To address these challenges, we propose a novel deep symbolic optimization learning framework that combines their advantages. Specifically, we focus on the node selection module within B&B solvers -- namely, deep symbolic optimization for node selection (Dso4NS). With data-driven approaches, Dso4NS guides the search for mathematical expressions within the high-dimensional discrete symbolic space and then incorporates the highest-performing mathematical expressions into a solver. The data-driven model captures the rich feature information in the input data and generates symbolic expressions, while the expressions deployed in solvers enable fast inference with high interpretability. Experiments demonstrate the effectiveness of Dso4NS in learning high-quality expressions, outperforming existing approaches on a CPU machine. Encouragingly, the learned CPU-based policies consistently achieve performance comparable to state-of-the-art GPU-based approaches.

Deep Symbolic Optimization for Combinatorial Optimization: Accelerating Node Selection by Discovering Potential Heuristics

TL;DR

This work addresses the challenge of efficiently solving NP-hard combinatorial optimization problems by learning node-selection heuristics for Branch-and-Bound solvers. It introduces Dso4NS, a deep symbolic optimization framework that uses a sequential symbolic-regression model to generate compact, human-readable expressions for node scoring, trained via reinforcement learning with a behavioral-cloning-like fitness signal. Experiments on three benchmark families demonstrate that CPU-based symbolic policies can outperform traditional baselines and rival GPU-based methods in solving time, node exploration, and PD integral, while preserving interpretability and fast inference. By marrying the strengths of handcrafted heuristics and data-driven learning, Dso4NS offers a practical, interpretable pathway to enhance solver efficiency without relying on heavy neural networks during deployment.

Abstract

Combinatorial optimization (CO) is one of the most fundamental mathematical models in real-world applications. Traditional CO solvers, such as Branch-and-Bound (B&B) solvers, heavily rely on expert-designed heuristics, which are reliable but require substantial manual tuning. Recent studies have leveraged deep learning (DL) models as an alternative to capture rich feature patterns for improved performance on GPU machines. Nonetheless, the drawbacks of high training and inference costs, as well as limited interpretability, severely hinder the adoption of DL methods in real-world applications. To address these challenges, we propose a novel deep symbolic optimization learning framework that combines their advantages. Specifically, we focus on the node selection module within B&B solvers -- namely, deep symbolic optimization for node selection (Dso4NS). With data-driven approaches, Dso4NS guides the search for mathematical expressions within the high-dimensional discrete symbolic space and then incorporates the highest-performing mathematical expressions into a solver. The data-driven model captures the rich feature information in the input data and generates symbolic expressions, while the expressions deployed in solvers enable fast inference with high interpretability. Experiments demonstrate the effectiveness of Dso4NS in learning high-quality expressions, outperforming existing approaches on a CPU machine. Encouragingly, the learned CPU-based policies consistently achieve performance comparable to state-of-the-art GPU-based approaches.
Paper Structure (35 sections, 7 equations, 1 figure, 7 tables, 1 algorithm)

This paper contains 35 sections, 7 equations, 1 figure, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. MILP and Branch-and-Bound Algorithm
  4. Node Selection Heuristics
  5. Symbolic Regression
  6. Method
  7. Expressions for Node Selection
  8. Generation of Mathematical Expressions
  9. Input Features
  10. Sequential Model
  11. Reinforcement Learning Formulation
  12. Fitness Measure
  13. Training
  14. Deployment
  15. Results
  16. ...and 20 more sections

Figures (1)

  • Figure 1: Illustration of the deep symbolic regression framework for heuristic discovery in the node selection module. Part 1 illustrates the NODECOMP heuristic for node selection in a B&B solver. Then, we aim to learn a better mathematical expression for the scoring function $g$. Part 2 is the generation process of a symbolic expression, in which we leverage a sequential model to iteratively choose tokens in a predefined token library. Part 3 shows the reinforcement-learning-based training algorithm for the expression generator.