Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design
Paul Saves, Edward Hallé-Hannan, Jasper Bussemaker, Youssef Diouane, Nathalie Bartoli
TL;DR
The paper tackles the challenge of optimizing complex systems with mixed-variable, hierarchical, and conditional design spaces. It advances a unified framework that combines design space graphs, meta/decree variable concepts, and hierarchical distances/kernels to enable surrogate modeling and Bayesian optimization on variable-size domains. The framework extends SMT with a generalized design-space graph, introduces a graph-structured kernel (and an Alg-Kernel) that remains SPD, and demonstrates through neural-network hyperparameter tuning and a green-aircraft DRAGON case study. The open-source SMT 2.0 implementation and real-world validations highlight the approach's potential to improve sample efficiency and decision-making in architecture design tasks, while outlining avenues for handling more complex graph structures and automated verification in future work.
Abstract
Simulation-based problems involving mixed-variable inputs frequently feature domains that are hierarchical, conditional, heterogeneous, or tree-structured. These characteristics pose challenges for data representation, modeling, and optimization. This paper reviews extensive literature on these structured input spaces and proposes a unified framework that generalizes existing approaches. In this framework, input variables may be continuous, integer, or categorical. A variable is described as meta if its value governs the presence of other decreed variables, enabling the modeling of conditional and hierarchical structures. We further introduce the concept of partially-decreed variables, whose activation depends on contextual conditions. To capture these inter-variable hierarchical relationships, we introduce design space graphs, combining principles from feature modeling and graph theory. This allows the definition of general hierarchical domains suitable for describing complex system architectures. Our framework defines hierarchical distances and kernels to enable surrogate modeling and optimization on hierarchical domains. We demonstrate its effectiveness on complex system design problems, including a neural network and a green-aircraft case study. Our methods are available in the open-source Surrogate Modeling Toolbox (SMT 2.0).