Landscape Features in Single-Objective Continuous Optimization: Have We Hit a Wall in Algorithm Selection Generalization?

TL;DR

The paper investigates whether state-of-the-art landscape-feature representations (ELA, TinyTLA, Doe2Vec, TransOptAS, DeepELA) can generalize AS models to unseen, out-of-distribution problems in single-objective continuous optimization. Using affine combinations of BBOB problems and multiple evaluation settings, it shows that feature-based AS models rarely outperform a simple Single Best Solver on out-of-distribution data, with Ela features often providing the strongest signal in easier settings. The study analyzes feature complementarity and the alignment between feature similarity and algorithm performance, finding that alignment occurs mainly at extremely high feature similarity, which explains limited generalization. It concludes that new directions—such as trajectory-based features and larger, more diverse benchmarks—are needed, and highlights the importance of testing AS methods on real-world problems when available.

Abstract

%% Text of abstract The process of identifying the most suitable optimization algorithm for a specific problem, referred to as algorithm selection (AS), entails training models that leverage problem landscape features to forecast algorithm performance. A significant challenge in this domain is ensuring that AS models can generalize effectively to novel, unseen problems. This study evaluates the generalizability of AS models based on different problem representations in the context of single-objective continuous optimization. In particular, it considers the most widely used Exploratory Landscape Analysis features, as well as recently proposed Topological Landscape Analysis features, and features based on deep learning, such as DeepELA, TransOptAS and Doe2Vec. Our results indicate that when presented with out-of-distribution evaluation data, none of the feature-based AS models outperform a simple baseline model, i.e., a Single Best Solver.

Figures (10)

• Figure 1: Distribution of the normalized precision scores in each algorithm portfolio. A lower normalized precision signifies a better performance of an algorithm inside a portfolio
• Figure 2: Spearman correlation clustermap between pairs of features from different feature groups
• Figure 3: Spearman correlation clustermap between the ela_scaled features and the other feature groups
• Figure 4: Cosine similarity of problem pairs, calculated with different feature sets. The first feature name in the title of each subplot refers to the horizontal axis, while the second feature name refers to the vertical axis.
• Figure 5: Distribution of feature and performance similarities for the 2DE+2PSO portfolio for 1,000,000 problem pairs