Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Denoising Diffusion-Based Evolutionary Algorithm Framework: Application to the Maximum Independent Set Problem

Joan Salvà Soler, Günther R. Raidl

TL;DR

A Denoising Diffusion-based Evolutionary Algorithm (DDEA) framework that synergistically integrates DDMs and EAs, highlighting the potential of hybridizing DDMs and EAs and offering a promising direction for the development of powerful machine learning solvers for complex combinatorial optimization problems.

Abstract

Denoising diffusion models (DDMs) offer a promising generative approach for combinatorial optimization, yet they often lack the robust exploration capabilities of traditional metaheuristics like evolutionary algorithms (EAs). We propose a Denoising Diffusion-based Evolutionary Algorithm (DDEA) framework that synergistically integrates these paradigms. It utilizes pre-trained DDMs for both high-quality and diverse population initialization and a novel diffusion-based recombination operator, trained via imitation learning against an optimal demonstrator. Evaluating DDEA on the Maximum Independent Set problem on Erdős-Rényi graphs, we demonstrate notable improvements over DIFUSCO, a leading DDM solver. DDEA consistently outperforms it given the same time budget, and surpasses Gurobi on larger graphs under the same time limit, with DDEA's solution sizes being 3.9% and 7.5% larger on the ER-300-400 and ER-700-800 datasets, respectively. In out-of-distribution experiments, DDEA provides solutions of 11.6% higher quality than DIFUSCO under the same time limit. Ablation studies confirm that both diffusion initialization and recombination are crucial. Our work highlights the potential of hybridizing DDMs and EAs, offering a promising direction for the development of powerful machine learning solvers for complex combinatorial optimization problems.

A Denoising Diffusion-Based Evolutionary Algorithm Framework: Application to the Maximum Independent Set Problem

TL;DR

A Denoising Diffusion-based Evolutionary Algorithm (DDEA) framework that synergistically integrates DDMs and EAs, highlighting the potential of hybridizing DDMs and EAs and offering a promising direction for the development of powerful machine learning solvers for complex combinatorial optimization problems.

Abstract

Denoising diffusion models (DDMs) offer a promising generative approach for combinatorial optimization, yet they often lack the robust exploration capabilities of traditional metaheuristics like evolutionary algorithms (EAs). We propose a Denoising Diffusion-based Evolutionary Algorithm (DDEA) framework that synergistically integrates these paradigms. It utilizes pre-trained DDMs for both high-quality and diverse population initialization and a novel diffusion-based recombination operator, trained via imitation learning against an optimal demonstrator. Evaluating DDEA on the Maximum Independent Set problem on Erdős-Rényi graphs, we demonstrate notable improvements over DIFUSCO, a leading DDM solver. DDEA consistently outperforms it given the same time budget, and surpasses Gurobi on larger graphs under the same time limit, with DDEA's solution sizes being 3.9% and 7.5% larger on the ER-300-400 and ER-700-800 datasets, respectively. In out-of-distribution experiments, DDEA provides solutions of 11.6% higher quality than DIFUSCO under the same time limit. Ablation studies confirm that both diffusion initialization and recombination are crucial. Our work highlights the potential of hybridizing DDMs and EAs, offering a promising direction for the development of powerful machine learning solvers for complex combinatorial optimization problems.

Paper Structure

This paper contains 24 sections, 1 equation, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Denoising Diffusion Models for Combinatorial Optimization.
  4. Evolutionary Algorithms.
  5. ML-EA Hybridization.
  6. The DDEA Framework
  7. The Maximum Independent Set Problem, its Solution Representation, and Solution Decoding
  8. Diffusion-Based Initialization
  9. Diffusion-Based Recombination
  10. Optimized Recombination as an Expert Demonstrator.
  11. The Diffusion Recombination Model Architecture.
  12. Training via Imitation Learning.
  13. Experimental Evaluation
  14. Setup
  15. Datasets.
  16. ...and 9 more sections

Figures (2)

  • Figure 1: Overview of the DDEA design in contrast to a classical EA. DDEA replaces standard initialization and recombination with diffusion-based counterparts.
  • Figure 2: Average cost (IS size, higher is better) against different runtime budgets for DDEA and DIFUSCO.