Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Context-aware Diversity Enhancement for Neural Multi-Objective Combinatorial Optimization

Yongfan Lu, Zixiang Di, Bingdong Li, Shengcai Liu, Hong Qian, Peng Yang, Ke Tang, Aimin Zhou

TL;DR

This work tackles multi-objective combinatorial optimization (MOCO) by introducing Context-aware Diversity Enhancement (CDE), which combines node-level sequence modeling with autoregressive node embeddings and solution-level hypervolume expectation maximization to couple preferences to diverse Pareto fronts. A Hypervolume Residual Update (HRU) strategy preserves both local and non-local Pareto information, while Explicit and Implicit Dual Inference (EI^2) paired with Local Subset Selection Acceleration (LSSA) enhances convergence and efficiency. The approach employs a hypernetwork to condition decoder parameters on polar-angle preferences, guiding solution generation toward high hypervolume with reduced duplicates. Across MOTSP, MOCVRP, and MOKP, CDE consistently outperforms state-of-the-art baselines in hypervolume and diversity, with competitive runtimes and strong generalization to larger instances.

Abstract

Multi-objective combinatorial optimization (MOCO) problems are prevalent in various real-world applications. Most existing neural MOCO methods rely on problem decomposition to transform an MOCO problem into a series of singe-objective combinatorial optimization (SOCO) problems and train attention models based on a single-step and deterministic greedy rollout. However, inappropriate decomposition and undesirable short-sighted behaviors of previous methods tend to induce a decline in diversity. To address the above limitation, we design a Context-aware Diversity Enhancement algorithm named CDE, which casts the neural MOCO problems as conditional sequence modeling via autoregression (node-level context awareness) and establishes a direct relationship between the mapping of preferences and diversity indicator of reward based on hypervolume expectation maximization (solution-level context awareness). Based on the solution-level context awareness, we further propose a hypervolume residual update strategy to enable the Pareto attention model to capture both local and non-local information of the Pareto set/front. The proposed CDE can effectively and efficiently grasp the context information, resulting in diversity enhancement. Experimental results on three classic MOCO problems demonstrate that our CDE outperforms several state-of-the-art baselines.

Context-aware Diversity Enhancement for Neural Multi-Objective Combinatorial Optimization

TL;DR

This work tackles multi-objective combinatorial optimization (MOCO) by introducing Context-aware Diversity Enhancement (CDE), which combines node-level sequence modeling with autoregressive node embeddings and solution-level hypervolume expectation maximization to couple preferences to diverse Pareto fronts. A Hypervolume Residual Update (HRU) strategy preserves both local and non-local Pareto information, while Explicit and Implicit Dual Inference (EI^2) paired with Local Subset Selection Acceleration (LSSA) enhances convergence and efficiency. The approach employs a hypernetwork to condition decoder parameters on polar-angle preferences, guiding solution generation toward high hypervolume with reduced duplicates. Across MOTSP, MOCVRP, and MOKP, CDE consistently outperforms state-of-the-art baselines in hypervolume and diversity, with competitive runtimes and strong generalization to larger instances.

Abstract

Multi-objective combinatorial optimization (MOCO) problems are prevalent in various real-world applications. Most existing neural MOCO methods rely on problem decomposition to transform an MOCO problem into a series of singe-objective combinatorial optimization (SOCO) problems and train attention models based on a single-step and deterministic greedy rollout. However, inappropriate decomposition and undesirable short-sighted behaviors of previous methods tend to induce a decline in diversity. To address the above limitation, we design a Context-aware Diversity Enhancement algorithm named CDE, which casts the neural MOCO problems as conditional sequence modeling via autoregression (node-level context awareness) and establishes a direct relationship between the mapping of preferences and diversity indicator of reward based on hypervolume expectation maximization (solution-level context awareness). Based on the solution-level context awareness, we further propose a hypervolume residual update strategy to enable the Pareto attention model to capture both local and non-local information of the Pareto set/front. The proposed CDE can effectively and efficiently grasp the context information, resulting in diversity enhancement. Experimental results on three classic MOCO problems demonstrate that our CDE outperforms several state-of-the-art baselines.
Paper Structure (55 sections, 1 theorem, 34 equations, 7 figures, 6 tables, 3 algorithms)

This paper contains 55 sections, 1 theorem, 34 equations, 7 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neural Heuristics for MOCO.
  4. Hypervolume Expectation Maximization.
  5. Sequence Modeling.
  6. Preliminaries
  7. Multi-Objective Combinatorial Optimization
  8. Performance Evaluation
  9. Methodology
  10. Overview
  11. Node-level Context Awareness via Sequence Modeling
  12. Solution-level Context Awareness via Hypervolume Expec- tation Maximization
  13. Hypervolume Residual Update Strategy
  14. Novel Inference Strategy
  15. Explicit and Implicit Dual Inference ($\rm{EI^2}$).
  16. ...and 40 more sections

Key Result

Lemma 4.1

$Y=\{\boldsymbol{y}^1,...,\boldsymbol{y}^n\}$ denotes set of finite objective vectors and $\boldsymbol{r} = (r_1,...r_m)$ denotes a reference point, which satisfies $\forall i \in \{1,2,...,m\}, \boldsymbol{r} \succ \boldsymbol{y}^i$. where $\Phi=\frac{2\pi^{m/2}}{\Gamma(m/2)}$ denotes the area of the $(m-1)$-D unit sphere, which is a dimension-specific constant and $\Gamma(x)=\int^{\inf}_0 z^{x-

Figures (7)

  • Figure 1: Pipeline of CDE. Left ( $p'$-th training iteration): CDE is trained using a context-aware strategy to establish the mapping between preferences (polar angles) and solutions with effective and efficient diversity enhancement. Specifically, the encoder takes in a sampled instance for projection and solutions generated from previous preferences $\boldsymbol{\theta}^{p‘-1}$ offer context information for the current preference $\theta$. In addition, $\theta$ generates the decoder parameters through a specifically designed HV network. At step $t$ in the decoder, a context embedding $\boldsymbol{g}_c$ is used to calculate the probability of node selection. The solutions derived from CDE are aligned precisely with the polar angles in a polar coordinate system under mild conditions. Right: during inference, CDE employs an explicit and implicit dual inference ($\rm{EI^2}$) approach to enhance convergence and diversity, and incorporates local subset selection acceleration (LSSA) to enhance efficiency. A well-trained Pareto attention model applies the $\rm{EI^2}$ approach to each polar angle, resulting in solutions with superior convergence and diversity, while LSSA facilitates efficient selection.
  • Figure 2: Visual comparisons on Bi-TSP20 (a-e) and Bi-KP50 (f-j). CDE has better adaptability than WS- and TCH-based approaches. CDE1 and CDE2 denote the CDE w/o node-level context awareness and solution-level context awareness, respectively. Any ablation of context-aware components leads to a decrease in diversity.
  • Figure 3: Pareto front hypervolume calculation in the polar coordinate. $\mathcal{G}(\boldsymbol{x},\theta)$ is the distance from the reference point to the Pareto front along angle $\theta$. $\mathcal{V}(\theta)$ is the projected distance at angle $\theta$.
  • Figure 4: Alternative Pareto front hypervolume calculation in the polar coordinate.
  • Figure 5: Hyperparameter study. (a) Effect of the maximal size of polar angles pool used in training on Bi-TSP50. (b) Effect of the maximal size of polar angles pool used in training on Tri-TSP100. (c) Effect of the control parameter of potential energy on Bi-TSP50. (d) Effect of the control parameter of potential energy on Tri-TSP50. points from new solutions for updating the Pareto front.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Definition 3.1: Pareto dominance yu1974cone
  • Definition 3.2: Pareto optimality
  • Definition 3.3: Pareto Set and Pareto Front
  • Lemma 4.1: Hypervolume scalarization of a finite set shang2018newdeng2019approximatingzhang2020random
  • Definition H.1: Neighborhood