Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Surrogate-Based Strategy for Accelerating Convergence Speed when Solving Expensive Unconstrained Multi-Objective Optimisation Problems

Tiwonge Msulira Banda, Alexandru-Ciprian Zăvoianu

TL;DR

This work tackles the bottleneck of solving expensive unconstrained MOOPs by introducing an adaptive surrogate accelerator within a two-loop MOEA framework. By training per-objective surrogates (GPR, 1D-CNN, RFR) on only the most recent generation and integrating surrogate solutions in a controlled, early fashion (50% transfer; activation from gen 2; adaptive deactivation), the approach significantly accelerates early convergence for NSGA-II and MOEA/D with modest overhead. Across 31 benchmark problems and a North-East Atlantic fish stock case study, the method yields faster early improvement in $Hv(PF_c)$ and generally maintains competitive end-of-run performance, outperforming a lightweight interpolation baseline in early phases. The results suggest practical benefits for industrial CI-MOOPs, enabling high-quality Pareto fronts with far fewer true evaluations and highlighting avenues for further enhancement via ensemble surrogates and broader solver comparisons.

Abstract

Multi-Objective Evolutionary Algorithms (MOEAs) have proven effective at solving Multi-Objective Optimisation Problems (MOOPs). However, their performance can be significantly hindered when applied to computationally intensive industrial problems. To address this limitation, we propose an adaptive surrogate modelling approach designed to accelerate the early-stage convergence speed of state-of-the-art MOEAs. This is important because it ensures that a solver can identify optimal or near-optimal solutions with relatively few fitness function evaluations, thereby saving both time and computational resources. Our method employs a two-loop architecture. The outer loop runs a (baseline) host MOEA which carries out true fitness evaluations. The inner loop contains an Adaptive Accelerator that leverages data-driven machine learning (ML) surrogate models to approximate fitness functions. Integrated with NSGA-II and MOEA/D, our approach was tested on 31 widely known benchmark problems and a real-world North Sea fish abundance modelling case study. The results demonstrate that by incorporating Gaussian Process Regression, one-dimensional Convolutional Neural Networks, and Random Forest Regression, our proposed approach significantly accelerates the convergence speed of MOEAs in the early phases of optimisation.

Adaptive Surrogate-Based Strategy for Accelerating Convergence Speed when Solving Expensive Unconstrained Multi-Objective Optimisation Problems

TL;DR

This work tackles the bottleneck of solving expensive unconstrained MOOPs by introducing an adaptive surrogate accelerator within a two-loop MOEA framework. By training per-objective surrogates (GPR, 1D-CNN, RFR) on only the most recent generation and integrating surrogate solutions in a controlled, early fashion (50% transfer; activation from gen 2; adaptive deactivation), the approach significantly accelerates early convergence for NSGA-II and MOEA/D with modest overhead. Across 31 benchmark problems and a North-East Atlantic fish stock case study, the method yields faster early improvement in and generally maintains competitive end-of-run performance, outperforming a lightweight interpolation baseline in early phases. The results suggest practical benefits for industrial CI-MOOPs, enabling high-quality Pareto fronts with far fewer true evaluations and highlighting avenues for further enhancement via ensemble surrogates and broader solver comparisons.

Abstract

Multi-Objective Evolutionary Algorithms (MOEAs) have proven effective at solving Multi-Objective Optimisation Problems (MOOPs). However, their performance can be significantly hindered when applied to computationally intensive industrial problems. To address this limitation, we propose an adaptive surrogate modelling approach designed to accelerate the early-stage convergence speed of state-of-the-art MOEAs. This is important because it ensures that a solver can identify optimal or near-optimal solutions with relatively few fitness function evaluations, thereby saving both time and computational resources. Our method employs a two-loop architecture. The outer loop runs a (baseline) host MOEA which carries out true fitness evaluations. The inner loop contains an Adaptive Accelerator that leverages data-driven machine learning (ML) surrogate models to approximate fitness functions. Integrated with NSGA-II and MOEA/D, our approach was tested on 31 widely known benchmark problems and a real-world North Sea fish abundance modelling case study. The results demonstrate that by incorporating Gaussian Process Regression, one-dimensional Convolutional Neural Networks, and Random Forest Regression, our proposed approach significantly accelerates the convergence speed of MOEAs in the early phases of optimisation.
Paper Structure (18 sections, 3 equations, 20 figures, 3 tables)

This paper contains 18 sections, 3 equations, 20 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Approach
  4. Data collection and pre-processing
  5. Model training and fitness function estimation
  6. Parameters
  7. Experimental Design
  8. Incorporation into Solvers
  9. Benchmark Problems
  10. Performance Indicator
  11. Results and Discussion
  12. Adaptive Surrogate Deactivation
  13. Mean Performance on MOOP Benchmark Set
  14. Statistical Analysis of MOOP Benchmark Set Performance
  15. Performance on Individual Problems and Problem Suites
  16. ...and 3 more sections

Figures (20)

  • Figure 1: Overview of the Adaptive Accelerated MOEA showing the two loops. The standard (host) MOEA is shown in black whereas the surrogate-based accelerator is shown in blue.
  • Figure 2: Mean performance of GPR-based Adaptive Accelerator for NSGA-II on DTLZ7 and MOEA/D-DRA on LZ09_F1 across the 100 independent runs. The shaded part is the mean interval the surrogate was active. The brown line indicates the mean value of $\chi_{t}$ -- the percentage of surrogate solutions from the penultimate generation that are in the archive at generation $t$ (read from the right y-axis).
  • Figure 3: Comparison with full-on surrogate version (i.e., no exit) for the RFR-NSGA-II variant.
  • Figure 4: Mean comparative performance (convergence and spread) of CNN-NSGA-II and GPR-NSGA-II variants with surrogate integration thresholds of 50%, 75% and 100%.
  • Figure 5: Mean comparative performance of the surrogate-enhanced NSGA-II and MOEA/D-DRA and their standard versions on all 31 problems using the hypervolume ($Hv(PF_c)$) metric.
  • ...and 15 more figures