Quantifying Individual and Joint Module Impact in Modular Optimization Frameworks

TL;DR

This paper addresses how individual modules and their interactions shape performance in modular optimization frameworks by applying functional-ANOVA (f-ANOVA) to two modular algorithms, modCMA and modDE, across the BBOB benchmark. Using 324 modCMA and 576 modDE variants over 24 problems and two dimensions, the authors quantify both individual and interaction effects (pairwise and triple) under suite-wide and problem-specific settings, revealing that module importance shifts with dimensionality and budget (e.g., weights_option and mirrored for modCMA at low $d$, base_sampler at high $d$; lpsr dominates modDE). The results show over 91% of variance can be explained by these effects, with modDE displaying a stronger move from individual to interaction effects as resources grow, while modCMA remains more driven by individual effects in higher dimensions. These insights aid algorithm designers in post-hoc analysis and module-space reduction, though computational cost limits higher-order interaction analysis. The work suggests extending to additional modular frameworks and alternative importance measures to broaden applicability.

Abstract

This study explores the influence of modules on the performance of modular optimization frameworks for continuous single-objective black-box optimization. There is an extensive variety of modules to choose from when designing algorithm variants, however, there is a rather limited understanding of how each module individually influences the algorithm performance and how the modules interact with each other when combined. We use the functional ANOVA (f-ANOVA) framework to quantify the influence of individual modules and module combinations for two algorithms, the modular Covariance Matrix Adaptation (modCMA) and the modular Differential Evolution (modDE). We analyze the performance data from 324 modCMA and 576 modDE variants on the BBOB benchmark collection, for two problem dimensions, and three computational budgets. Noteworthy findings include the identification of important modules that strongly influence the performance of modCMA, such as the~\textit{weights\ option} and~\textit{mirrored} modules for low dimensional problems, and the~\textit{base\ sampler} for high dimensional problems. The large individual influence of the~\textit{lpsr} module makes it very important for the performance of modDE, regardless of the problem dimensionality and the computational budget. When comparing modCMA and modDE, modDE undergoes a shift from individual modules being more influential, to module combinations being more influential, while modCMA follows the opposite pattern, with an increase in problem dimensionality and computational budget.

Figures (3)

• Figure 1: Variance in the algorithm performance explained by the individual effects of the algorithm modules of modCMA and modDE, in problem dimension $d=5$ and $d=30$ and for budgets of 100$d$, 500$d$, and 1500$d$ function evaluations.
• Figure 2: Variance (%) in the algorithm performance explained by the pairwise effects of the algorithm modules for a,c) modCMA and b,d) modDE in 5 and 30 dimensions respectively, for a budget of 500$d$.
• Figure 3: Cosine similarity between three BBOB problems represented by the individual, pairwise, and triplet module effects for a,c) modCMA and b,d) modDE for a budget of 500$d$.