PyPop7: A Pure-Python Library for Population-Based Black-Box Optimization

TL;DR

PyPop7 addresses large-scale, gradient-free optimization by offering a pure-Python, modular library with a unified API for population-based optimizers, augmented by repeatability reports and benchmarking against established platforms. It highlights performance-focused implementations using high-performance libraries, extensive cross-method benchmarking on $2000$-dimensional and rotated/shifted problems, and diverse task-based evaluations to justify the need for multiple optimizer variants. The paper contributes a practical framework, automated testing, and case studies that demonstrate usability and competitive efficiency, with clear paths for future enhancements. This work enables researchers and practitioners to systematically compare, extend, and apply large-scale BBO methods across real-world, high-dimensional problems.

Abstract

In this paper, we present an open-source pure-Python library called PyPop7 for black-box optimization (BBO). As population-based methods (e.g., evolutionary algorithms, swarm intelligence, and pattern search) become increasingly popular for BBO, the design goal of PyPop7 is to provide a unified API and elegant implementations for them, particularly in challenging high-dimensional scenarios. Since these population-based methods easily suffer from the notorious curse of dimensionality owing to random sampling as one of core operations for most of them, recently various improvements and enhancements have been proposed to alleviate this issue more or less mainly via exploiting possible problem structures: such as, decomposition of search distribution or space, low-memory approximation, low-rank metric learning, variance reduction, ensemble of random subspaces, model self-adaptation, and fitness smoothing. These novel sampling strategies could better exploit different problem structures in high-dimensional search space and therefore they often result in faster rates of convergence and/or better qualities of solution for large-scale BBO. Now PyPop7 has covered many of these important advances on a set of well-established BBO algorithm families and also provided an open-access interface to adding the latest or missed black-box optimizers for further functionality extensions. Its well-designed source code (under GPL-3.0 license) and full-fledged online documents (under CC-BY 4.0 license) have been freely available at \url{https://github.com/Evolutionary-Intelligence/pypop} and \url{https://pypop.readthedocs.io}, respectively.

Figures (9)

• Figure 1: A conceptual framework of PyPop7 for black-box optimization (BBO), where all optimizers colored in orange are mainly designed for large-scale BBO.
• Figure 2: Median comparisons of function evaluations and solution qualities of one baseline version https://pypop.readthedocs.io/en/latest/es/saes.html of https://pypop.readthedocs.io/en/latest/es/es.html from our library and the widely-used https://github.com/DEAP/deap library (under the same runtime for a fair comparison). Note that each of these two implementation versions is independently run 10 times on this 2000-dimensional, light-weighted test function https://github.com/Evolutionary-Intelligence/pypop/blob/main/pypop7/benchmarks/base_functions.py. Here we do not use the standard rotation-shift operations, different from the following computationally-expensive benchmarking process (of quadratic complexity), in order to generate light-weighted function evaluations (of only linear complexity) even in high dimensions.
• Figure 3: Median comparisons of function evaluations and solution qualities between three large-scale ES versions of our library and https://github.com/DEAP/deap's CMA-ES. The experimental settings are the same as Figure \ref{['fig:median_compare_pypop7_deap_3']} (given the maximal runtime: 3 hours).
• Figure 4: Median comparisons of function evaluations and solution qualities of PSO, EDA, and DE between our library and the widely-used https://github.com/DEAP/deap library. The experimental settings are the same as Figure \ref{['fig:median_compare_pypop7_deap_3']} (given the maximal runtime: 3 hours).
• Figure 5: Median convergence rate comparisons of 7 PSO versions on 20 high-dimensional computationally-expensive test functions (with the standard rotation-and-shift operations of quadratic complexity for benchmarking).