Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A GPU Implementation of Multi-Guiding Spark Fireworks Algorithm for Efficient Black-Box Neural Network Optimization

Xiangrui Meng, Ying Tan

TL;DR

Black-box neural network optimization in high-dimensional spaces is challenging due to the lack of gradient information. The authors present GPU-MGFWA, a CUDA/PyTorch implementation that parallelizes fireworks and sparks across $B$ batches with tensor shapes such as $F: B×N×D$, $S_e: B×N×D$, $S_g: B×N×D$, and $Y: B×N×D$. Empirical results show GPU-MGFWA delivers faster convergence and lower computation time than the CPU baseline while achieving comparable or better solution quality, with larger networks yielding larger speedups. This work demonstrates that GPU-accelerated swarm intelligence can scale to real-time, large-scale black-box optimization tasks and broadens practical applicability across data-driven engineering and ML contexts.

Abstract

Swarm intelligence optimization algorithms have gained significant attention due to their ability to solve complex optimization problems. However, the efficiency of optimization in large-scale problems limits the use of related methods. This paper presents a GPU-accelerated version of the Multi-Guiding Spark Fireworks Algorithm (MGFWA), which significantly improves the computational efficiency compared to its traditional CPU-based counterpart. We benchmark the GPU-MGFWA on several neural network black-box optimization problems and demonstrate its superior performance in terms of both speed and solution quality. By leveraging the parallel processing power of modern GPUs, the proposed GPU-MGFWA results in faster convergence and reduced computation time for large-scale optimization tasks. The proposed implementation offers a promising approach to accelerate swarm intelligence algorithms, making them more suitable for real-time applications and large-scale industrial problems. Source code is released at https://github.com/mxxxr/MGFWA.

A GPU Implementation of Multi-Guiding Spark Fireworks Algorithm for Efficient Black-Box Neural Network Optimization

TL;DR

Black-box neural network optimization in high-dimensional spaces is challenging due to the lack of gradient information. The authors present GPU-MGFWA, a CUDA/PyTorch implementation that parallelizes fireworks and sparks across batches with tensor shapes such as , , , and . Empirical results show GPU-MGFWA delivers faster convergence and lower computation time than the CPU baseline while achieving comparable or better solution quality, with larger networks yielding larger speedups. This work demonstrates that GPU-accelerated swarm intelligence can scale to real-time, large-scale black-box optimization tasks and broadens practical applicability across data-driven engineering and ML contexts.

Abstract

Swarm intelligence optimization algorithms have gained significant attention due to their ability to solve complex optimization problems. However, the efficiency of optimization in large-scale problems limits the use of related methods. This paper presents a GPU-accelerated version of the Multi-Guiding Spark Fireworks Algorithm (MGFWA), which significantly improves the computational efficiency compared to its traditional CPU-based counterpart. We benchmark the GPU-MGFWA on several neural network black-box optimization problems and demonstrate its superior performance in terms of both speed and solution quality. By leveraging the parallel processing power of modern GPUs, the proposed GPU-MGFWA results in faster convergence and reduced computation time for large-scale optimization tasks. The proposed implementation offers a promising approach to accelerate swarm intelligence algorithms, making them more suitable for real-time applications and large-scale industrial problems. Source code is released at https://github.com/mxxxr/MGFWA.
Paper Structure (12 sections, 1 equation, 4 figures, 2 tables, 1 algorithm)

This paper contains 12 sections, 1 equation, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fireworks Algorithm and its Variants
  4. GPU-Based Implementation of Swarm Intelligence Algorithms
  5. Implementation of GPU-MGFWA
  6. Experiments
  7. Testbeds and Experimental Setup
  8. Testbeds:
  9. Experimental Setup:
  10. Main Results
  11. Conclusion
  12. Acknowledgments.

Figures (4)

  • Figure 1: Comparison of the two algorithms for explosion operation. MGFWA: sparks are generated and evaluated for each firework, all fireworks are processed serially. GPU-MGFWA: sparks are generated and evaluated for all fireworks at the same time, all fireworks are processed in parallel.
  • Figure 2: Optimization Efficiency Comparison between MGFWA and GPU-MGFWA on small scale Black-Box Neural Network Optimization
  • Figure 3: Optimization Efficiency Comparison between MGFWA and GPU-MGFWA on medium scale Black-Box Neural Network Optimization
  • Figure 4: Optimization Efficiency Comparison between MGFWA and GPU-MGFWA on large scale Black-Box Neural Network Optimization