Impact of spatial transformations on landscape features of CEC2022 basic benchmark problems

TL;DR

The paper investigates how spatial transformations applied to five CEC2022 basic benchmark problems affect low-level landscape features captured by Exploratory Landscape Analysis (ELA). Using 55 ELA features and a large, controlled perturbation protocol (translations, scalings, and rotations on inputs and outputs), it demonstrates that transformations can significantly alter feature distributions, as measured by KS tests and Earth Mover's Distance, and visualizes these shifts with 2D UMAP projections. The findings reveal that domain translations and objective-value scaling can cause substantial changes in many features, while rotations have more limited impact; PCA-based features are notably more robust, though not invariant. The results highlight a need for careful, principled instance generation and normalization in benchmarking, and encourage further research into the robustness of ELA-derived insights for algorithm selection and performance prediction.

Abstract

When benchmarking optimization heuristics, we need to take care to avoid an algorithm exploiting biases in the construction of the used problems. One way in which this might be done is by providing different versions of each problem but with transformations applied to ensure the algorithms are equipped with mechanisms for successfully tackling a range of problems. In this paper, we investigate several of these problem transformations and show how they influence the low-level landscape features of a set of 5 problems from the CEC2022 benchmark suite. Our results highlight that even relatively small transformations can significantly alter the measured landscape features. This poses a wider question of what properties we want to preserve when creating problem transformations, and how to fairly measure them.

Figures (6)

• Figure 1: Landscapes of CEC2022 basic problems in $[-100, 100]^2$.
• Figure 2: Examples of instance generation via spatial transformations applied to one of the original CEC2022 benchmark problems, shown here in two dimensions, with optima locations marked by crosses.
• Figure 3: UMAP-based projections of ELA features for various spatial transformations of five basic problems. The projection model is computed based on original problems and is applied to all samples shown in these figures. 100 samples of each original problem are shown with round (faded) coloured symbols, while samples of the transformed problems correspond to other coloured shapes, with different transformation magnitudes denoted by different symbol sizes. Each point represents a set of ELA features calculated after a Latin hypercube sampling in the search space. Only randomly subselected 4% of the transformed problem samples are presented in the graph to avoid overwhelming information with large data volumes.
• Figure 4: Results of applying four types of transformations to the base functions. EMD (dashed lines) and the number of features (solid lines) rejected by the KS-test between the original and transformed features via translation (left column) or scaling (right column) applied to $\boldsymbol{x}$ (top row) or $y$ (bottom row). Different colours represent different base problems. The EMD results of different problems are calculated based on normalized feature values.
• Figure 5: Features of instances rotated on $\boldsymbol{x}$, per problem. Only features where $diff_{ij}\geq 1\%$ on at least one problem are shown.