A Generalized and Configurable Benchmark Generator for Continuous Unconstrained Numerical Optimization

TL;DR

The paper introduces the Generalized Numerical Benchmark Generator (GNBG), a configurable tool for box-constrained, single-objective continuous optimization that uses a single parametric baseline function to produce diverse problem landscapes. By adjusting components, rotation structures, nonlinear transformations, and conditioning, GNBG can systematically vary modality, local-optima morphology, separability, and difficulty, enabling isolated or combined studies of algorithm performance. Key contributions include the Baseline Mathematical Formulation, a detailed Parameter Sensitivity Analysis, and a framework for generating targeted problem instances, including a 24-task GNBG test suite complemented by illustrative experiments. The work advances robust benchmarking by offering controllable, scalable, and accessible test environments to evaluate optimization methods under realistic and varied challenges, with future work to broaden empirical validation across more algorithms.

Abstract

As optimization challenges continue to evolve, so too must our tools and understanding. To effectively assess, validate, and compare optimization algorithms, it is crucial to use a benchmark test suite that encompasses a diverse range of problem instances with various characteristics. Traditional benchmark suites often consist of numerous fixed test functions, making it challenging to align these with specific research objectives, such as the systematic evaluation of algorithms under controllable conditions. This paper introduces the Generalized Numerical Benchmark Generator (GNBG) for singleobjective, box-constrained, continuous numerical optimization. Unlike the commonly used test suites that rely on multiple baseline functions and transformations, GNBG utilizes a single, parametric, and configurable baseline function. This design allows for control over various problem characteristics. Researchers using GNBG can generate instances that cover a broad range of morphological features, from unimodal to highly multimodal functions, various local optima patterns, and symmetric to highly asymmetric structures. The generated problems can also vary in separability, variable interaction structures, dimensionality, conditioning, and basin shapes. These customizable features enable the systematic evaluation and comparison of optimization methods, allowing researchers to examine the strengths and weaknesses of algorithms under diverse and controllable conditions.

Figures (7)

• Figure 1: Impact of $\lambda$ values on the morphology of a component generated by GNBG. For these illustrative examples, we set $d=2$, $o=1$, $\bm\mu=(0,0)$, $\bm\omega=(0,0,0,0)$, $\sigma=0$, $\mathbf{m}=(0,0)$, $\mathbf{H}=\mathbf{I}_{2 \times 2}$, and $\mathbf{R}=\mathbf{I}_{2 \times 2}$. Additionally, the 2-dimensional problem space is bounded to [-100,100] in each dimension. For a component generated by GNBG, $\lambda < 0.5$ yields a sub-linear basin, $\lambda = 0.5$ yields a linear basin, and $\lambda > 0.5$ yields a super-linear basin.
• Figure 2: Impact of $\mathbf{H}$ values on the morphology of a component generated by GNBG. For these illustrative examples, we set $d=2$, $o=1$, $\bm\mu=(0,0)$, $\bm\omega=(0,0,0,0)$, $\sigma=0$, $\mathbf{m}=(0,0)$, $\lambda=0.25$, and $\mathbf{R}=\mathbf{I}_{2 \times 2}$. Additionally, the 2-dimensional problem space is bounded to [-100,100] in each dimension.
• Figure 3: Examples of how, by configuring $\bm\Theta$, GNBG can generate desired variable interaction structures in an 8-dimensional component. We assume that the component is initially fully-separable and rotation-dependent. The variable interaction graphs associated with each matrix $\bm\Theta$ are illustrated in the right column.
• Figure 4: Impact of rotating the projection of $\mathbf{x}$ in the basin of a component onto the $x_{1}–x_{2}$ plane with different angles $\bm\Theta(1,2)$. For generating these illustrative examples, we set $d=2$, $o=1$, $\bm\mu=(0,0)$, $\bm\omega=(0,0,0,0)$, $\sigma=0$, $\mathbf{m}=(0,0)$, $\lambda=0.25$, $\mathbf{H}=\mathrm{diag}(1,10)$, and $\mathbf{R}$ is obtained by Algorithm \ref{['alg:RotationControlled']} based on the given angle $\bm\Theta(1,2)$. Additionally, the 2-dimensional problem space is bounded to [-100,100] in each dimension.
• Figure 5: Impact of $\bm\mu$ and $\bm\omega$ values on a 2-dimensional component generated by GNBG. For these illustrative examples, the 2-dimensional problem space is bounded to [-100,100] in each dimension, $o=1$, $\sigma=0$, $\mathbf{m}=(0,0)$, $\lambda=0.25$, $\mathbf{H}=\mathrm{diag}(1,1)$, and $\mathbf{R}=\mathbf{I}_{2 \times 2}$. This configuration focuses on the transformation impact and simplifies GNBG to $f(\mathbf{x}) = \left( \mathbb{T}(\mathbf{x}^\top) \mathbb{T} (\mathbf{x}) \right)^{\lambda}$.