Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Constrained Combinatorial Optimization leveraging Hypergraph Neural Networks

Nasimeh Heydaribeni, Xinrui Zhan, Ruisi Zhang, Tina Eliassi-Rad, Farinaz Koushanfar

TL;DR

HypOp offers a scalable, unsupervised framework for constrained combinatorial optimization by modeling problems as constraint hypergraphs and learning a continuous relaxation through a HyperGNN, $p=\mathcal{G}(\sigma;W)$, with a penalized objective and SA-based discretization. The approach enables distributed and parallel training, transfer learning on the same hypergraph, and a refinement stage via simulated annealing to boost accuracy. Across hypergraph MaxCut, MIS, SAT, and resource allocation benchmarks, HypOp consistently outperforms generic unsupervised solvers and several graph-based baselines, while delivering notable runtime advantages, including large-scale multi-GPU scalability. The results demonstrate HypOp’s potential for scientific discovery and large-scale optimization, including a drug-substance hypergraph application (NDC MaxCut), and highlight practical considerations such as architecture choice and the value of combining learning-based transforms with classical optimization.

Abstract

Scalable addressing of high dimensional constrained combinatorial optimization problems is a challenge that arises in several science and engineering disciplines. Recent work introduced novel application of graph neural networks for solving quadratic-cost combinatorial optimization problems. However, effective utilization of models such as graph neural networks to address general problems with higher order constraints is an unresolved challenge. This paper presents a framework, HypOp, which advances the state of the art for solving combinatorial optimization problems in several aspects: (i) it generalizes the prior results to higher order constrained problems with arbitrary cost functions by leveraging hypergraph neural networks; (ii) enables scalability to larger problems by introducing a new distributed and parallel training architecture; (iii) demonstrates generalizability across different problem formulations by transferring knowledge within the same hypergraph; (iv) substantially boosts the solution accuracy compared with the prior art by suggesting a fine-tuning step using simulated annealing; (v) shows a remarkable progress on numerous benchmark examples, including hypergraph MaxCut, satisfiability, and resource allocation problems, with notable run time improvements using a combination of fine-tuning and distributed training techniques. We showcase the application of HypOp in scientific discovery by solving a hypergraph MaxCut problem on NDC drug-substance hypergraph. Through extensive experimentation on various optimization problems, HypOp demonstrates superiority over existing unsupervised learning-based solvers and generic optimization methods.

Distributed Constrained Combinatorial Optimization leveraging Hypergraph Neural Networks

TL;DR

HypOp offers a scalable, unsupervised framework for constrained combinatorial optimization by modeling problems as constraint hypergraphs and learning a continuous relaxation through a HyperGNN, , with a penalized objective and SA-based discretization. The approach enables distributed and parallel training, transfer learning on the same hypergraph, and a refinement stage via simulated annealing to boost accuracy. Across hypergraph MaxCut, MIS, SAT, and resource allocation benchmarks, HypOp consistently outperforms generic unsupervised solvers and several graph-based baselines, while delivering notable runtime advantages, including large-scale multi-GPU scalability. The results demonstrate HypOp’s potential for scientific discovery and large-scale optimization, including a drug-substance hypergraph application (NDC MaxCut), and highlight practical considerations such as architecture choice and the value of combining learning-based transforms with classical optimization.

Abstract

Scalable addressing of high dimensional constrained combinatorial optimization problems is a challenge that arises in several science and engineering disciplines. Recent work introduced novel application of graph neural networks for solving quadratic-cost combinatorial optimization problems. However, effective utilization of models such as graph neural networks to address general problems with higher order constraints is an unresolved challenge. This paper presents a framework, HypOp, which advances the state of the art for solving combinatorial optimization problems in several aspects: (i) it generalizes the prior results to higher order constrained problems with arbitrary cost functions by leveraging hypergraph neural networks; (ii) enables scalability to larger problems by introducing a new distributed and parallel training architecture; (iii) demonstrates generalizability across different problem formulations by transferring knowledge within the same hypergraph; (iv) substantially boosts the solution accuracy compared with the prior art by suggesting a fine-tuning step using simulated annealing; (v) shows a remarkable progress on numerous benchmark examples, including hypergraph MaxCut, satisfiability, and resource allocation problems, with notable run time improvements using a combination of fine-tuning and distributed training techniques. We showcase the application of HypOp in scientific discovery by solving a hypergraph MaxCut problem on NDC drug-substance hypergraph. Through extensive experimentation on various optimization problems, HypOp demonstrates superiority over existing unsupervised learning-based solvers and generic optimization methods.
Paper Structure (35 sections, 7 equations, 12 figures, 6 tables, 2 algorithms)

This paper contains 35 sections, 7 equations, 12 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Paper Structure
  3. Problem Statement
  4. HypOp Method
  5. Hypergraph Modeling
  6. HyperGNN as the Learnable Transformation Function
  7. Mapping with Simulated Annealing
  8. Distributed and Parallel HypOp Training
  9. Distributed Training
  10. Parallel Multi-GPU Training
  11. Experiments
  12. Baselines
  13. Hypergraph MaxCut
  14. Maximum Independent Set Problem
  15. Parallel and Distributed Training
  16. ...and 20 more sections

Figures (12)

  • Figure 1: Hypergraph Modeling and Distributed Training of HyperGNN in HypOp
  • Figure 1: HypOp vs. Bipartite GNN. Comparison of HypOp with the bipartite GNN baseline for hypergraph MaxCut problem on synthetic random hypergraphs. For almost the same performance (a), HypOp has a remarkably less run time compared to the bipartite GNN baseline (b). HypOp performance is presented as the average of the results from 10 sets of experiments, with the error region showing the standard deviation of the results.
  • Figure 2: The overview of HypOp for solving problem \ref{['eq:problem']}.
  • Figure 2: Transfer Learning. Comparison of HypOp with and without Transfer Learning from MaxCut to MIS problem on random regular graphs with $d=3$. For almost the same performance (a), transfer learning provides the results in almost no amount of time compared to vanilla training (b).
  • Figure 2: HyperGCN vs. HyperGAT. Performance of HypOp with HyperGCN architecture, compared with HyperGAT for hypergraph MaxCut problem on synthetic random hypergraphs. HyperGAT can not achieve the same performance compared to HyperGCN, while requiring significantly more time to train. HypOp performance is presented as the average of the results from 10 sets of experiments and the error region shows the standard deviation of the results.
  • ...and 7 more figures