Combining GPU and CPU for accelerating evolutionary computing workloads

TL;DR

This study evaluates accelerating evolutionary computing workloads using CPU and GPU resources on MuJoCo-based simulations within Revolve2, and introduces a dynamic CPU–GPU hybrid allocator to exploit idle hardware. Across multiple models and variant counts, CPU dominance is observed, with GPU performance and variability highly sensitive to workload and model complexity. A hybrid strategy that partitions variants between CPU and GPU based on prior benchmarks can approach or surpass single-hardware runs at high workloads, though overhead and memory constraints limit universal gains. The work highlights the potential of dynamic resource management and CPU-cluster scaling as practical avenues to accelerate EC experiments on consumer devices and workstations.

Abstract

Evolutionary computing (EC) has proven to be effective in solving complex optimization and robotics problems. Unfortunately, typical Evolutionary Algorithms (EAs) are constrained by the computational capacity available to researchers. More recently, GPUs have been extensively used in speeding up workloads across a variety of fields in AI. This led us to the idea of considering utilizing GPUs for optimizing ECs, particularly for complex problems such as the evolution of artificial creatures in physics simulations. In this study, we compared the CPU and GPU performance across various simulation models, from simple box environments to more complex models. Additionally, we create and investigate a novel hybrid CPU + GPU scheme that aims to fully utilize the idle hardware capabilities present on most consumer devices. The strategy involves running simulation workloads on both the GPU and the CPU, dynamically adjusting the distribution of workload between the CPU and the GPU based on benchmark results. Our findings suggest that while the CPU demonstrates superior performance under most conditions, the hybrid CPU + GPU strategy shows promise at higher workloads. However, overall performance improvement is highly sensitive to simulation parameters such as the number of variants, the complexity of the model, and the duration of the simulation. These results demonstrate the potential of creative, dynamic resource management for experiments running physics simulations on workstations and consumer devices that have both GPUs and CPUs present.

Figures (6)

• Figure 1: Profile statistics of runtime of an Evolutionary Algorithm
• Figure 2: CPU vs. GPU execution time for different simulation models across a range of variants. Shaded regions represent the 95% confidence interval after 3 repetitions.
• Figure 3: GPU utilization trends across different simulation models as a function of the number of variants. Color indicates GPU utilization (%).
• Figure 4: GPU Time vs. Number of Steps for 124 variants (32, 128, 256, 512, 1024, 2056, 4096, 8192, 16384, 32768)
• Figure 5: Flowchart of the hybrid CPU-GPU workload distribution strategy. Tasks are allocated based on performance metrics gathered during initial benchmarking of running CPU and GPU individually and comparing the ratio of the run times.