Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization

Robbert Reijnen, Yaoxin Wu, Zaharah Bukhsh, Yingqian Zhang

TL;DR

This work introduces GS-MODAC, a Graph Neural Network-augmented DRL framework for dynamic algorithm configuration in multi-objective combinatorial optimization (MOCO). By representing the search state as a graph of normalized objective values and learning state embeddings with GNNs, GS-MODAC uses a PPO-based policy to adapt evolutionary algorithm parameters during search, aiming to improve convergence and Pareto diversity. The approach is instantiated on NSGA-II and MOPSO and evaluated on Flexible Job Shop Scheduling (FJSP) and Capacitated Vehicle Routing Problem (CVRP) across 2–5 objectives, demonstrating superior hypervolume performance and strong generalization to unseen sizes and more complex problem variants, with instance-agnostic rewards guiding learning. The results suggest GS-MODAC offers robust, scalable dynamic configuration for MOCO and generalizes across problem types, sizes, and objective configurations, potentially extending to a broad class of MOEAs.

Abstract

Deep reinforcement learning (DRL) has been widely used for dynamic algorithm configuration, particularly in evolutionary computation, which benefits from the adaptive update of parameters during the algorithmic execution. However, applying DRL to algorithm configuration for multi-objective combinatorial optimization (MOCO) problems remains relatively unexplored. This paper presents a novel graph neural network (GNN) based DRL to configure multi-objective evolutionary algorithms. We model the dynamic algorithm configuration as a Markov decision process, representing the convergence of solutions in the objective space by a graph, with their embeddings learned by a GNN to enhance the state representation. Experiments on diverse MOCO challenges indicate that our method outperforms traditional and DRL-based algorithm configuration methods in terms of efficacy and adaptability. It also exhibits advantageous generalizability across objective types and problem sizes, and applicability to different evolutionary computation methods.

Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization

TL;DR

This work introduces GS-MODAC, a Graph Neural Network-augmented DRL framework for dynamic algorithm configuration in multi-objective combinatorial optimization (MOCO). By representing the search state as a graph of normalized objective values and learning state embeddings with GNNs, GS-MODAC uses a PPO-based policy to adapt evolutionary algorithm parameters during search, aiming to improve convergence and Pareto diversity. The approach is instantiated on NSGA-II and MOPSO and evaluated on Flexible Job Shop Scheduling (FJSP) and Capacitated Vehicle Routing Problem (CVRP) across 2–5 objectives, demonstrating superior hypervolume performance and strong generalization to unseen sizes and more complex problem variants, with instance-agnostic rewards guiding learning. The results suggest GS-MODAC offers robust, scalable dynamic configuration for MOCO and generalizes across problem types, sizes, and objective configurations, potentially extending to a broad class of MOEAs.

Abstract

Deep reinforcement learning (DRL) has been widely used for dynamic algorithm configuration, particularly in evolutionary computation, which benefits from the adaptive update of parameters during the algorithmic execution. However, applying DRL to algorithm configuration for multi-objective combinatorial optimization (MOCO) problems remains relatively unexplored. This paper presents a novel graph neural network (GNN) based DRL to configure multi-objective evolutionary algorithms. We model the dynamic algorithm configuration as a Markov decision process, representing the convergence of solutions in the objective space by a graph, with their embeddings learned by a GNN to enhance the state representation. Experiments on diverse MOCO challenges indicate that our method outperforms traditional and DRL-based algorithm configuration methods in terms of efficacy and adaptability. It also exhibits advantageous generalizability across objective types and problem sizes, and applicability to different evolutionary computation methods.

Paper Structure

This paper contains 19 sections, 3 equations, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Multi-Objective Optimization (MOO) and Combinatorial Optimization (MOCO).
  4. Algorithm Configuration.
  5. The Method
  6. MDP formulation for GS-MODAC
  7. Graph-based policy learning and Training Algorithm
  8. Multi-Objective Evolutionary Algorithm Deployment
  9. Experiments
  10. Experimental results
  11. RQ3: How effectively can trained GS-MODAC models handle previously unseen, more complex problem variants?
  12. Conclusion
  13. Test problem configurations
  14. Multi-Objective Algorithms for MOCO
  15. Alternative Performance Metrics
  16. ...and 4 more sections

Figures (3)

  • Figure 1: The GS-MODAC framework. The framework integrates a DRL agent with a graph-based representation of the search state to dynamically configure each iteration of the search for multi-objective optimization. At each iteration, the current population is converted into a graph where nodes represent normalized objective values of solutions across multiple objective planes, and edges connect solutions within the different Pareto fronts. A Graph Neural Network (GNNs) extracts an embedding, which the agents uses configure the next iteration. The chosen actions are applied to the environment, which returns a new population of solutions, an updated state, and a reward signal for learning.
  • Figure 2: Comparison of GS-MODAC, SMAC3, and NSGA-II solution methods: (a) Average convergence rates and (b) Pareto front distributions.
  • Figure 3: Chromosome Representation FJSP MOGA zhang2011effective