Scaling Behaviors of Evolutionary Algorithms on GPUs: When Does Parallelism Pay Off?

TL;DR

This work analyzes how GPU parallelism affects evolutionary algorithms beyond mere speedups, revealing that performance gains are highly dependent on algorithmic structure and problem settings. By benchmarking 16 EAs across 30 numerical and neuroevolution tasks under fixed-FE and fixed-time regimes, the study shows GPU acceleration can alter convergence, diversity dynamics, and scalability in nontrivial ways. It highlights distinct scaling regimes with problem dimensionality $D$ and population size $N$, demonstrates that large GPU-enabled populations expose behavior unseen under CPU constraints, and argues for time-based evaluation and hardware-aware design in EA research. The findings challenge traditional benchmarking practices and emphasize the importance of considering parallel hardware when evaluating, designing, and deploying EAs on modern computing platforms.

Abstract

Evolutionary algorithms (EAs) are increasingly implemented on graphics processing units (GPUs) to leverage parallel processing capabilities for enhanced efficiency. However, existing studies largely emphasize the raw speedup obtained by porting individual algorithms from CPUs to GPUs. Consequently, these studies offer limited insight into when and why GPU parallelism fundamentally benefits EAs. To address this gap, we investigate how GPU parallelism alters the behavior of EAs beyond simple acceleration metrics. We conduct a systematic empirical study of 16 representative EAs on 30 benchmark problems. Specifically, we compare CPU and GPU executions across a wide range of problem dimensionalities and population sizes. Our results reveal that the impact of GPU acceleration is highly heterogeneous and depends strongly on algorithmic structure. We further demonstrate that conventional fixed-budget evaluation based on the number of function evaluations (FEs) is inadequate for GPU execution. In contrast, fixed-time evaluation uncovers performance characteristics that are unobservable under small or practically constrained FE budgets, particularly for adaptive and exploration-oriented algorithms. Moreover, we identify distinct scaling regimes in which GPU parallelism is beneficial, saturates, or degrades as problem dimensionality and population size increase. Crucially, we show that large populations enabled by GPUs not only improve hardware utilization but also reveal algorithm-specific convergence and diversity dynamics that are difficult to observe under CPU-constrained settings. Consequently, our findings indicate that GPU parallelism is not strictly an implementation detail, but a pivotal factor that influences how EAs should be evaluated, compared, and designed for modern computing platforms.

Figures (125)

• Figure 1: Number of publications retrieved over the past 30 years from the Web of Science using different requests. TS queries the topic field (title, abstract, and keywords), whereas TI restricts the keyword to the article title only.
• Figure 2: Overview of the experimental setup, including evaluated algorithms, benchmark problems, and hardware specifications.
• Figure 3: Evaluation methodology used in this study: the left panel depicts the classic EA workflow, while the right panel visualizes the resulting Pareto front of performance that contrasts computational efficiency (vertical axis) with solution quality (horizontal axis) for different algorithm settings.
• Figure 4: $f_{a_{2}}$
• Figure 5: $f_{a_{10}}$