SMT-EX: An Explainable Surrogate Modeling Toolbox for Mixed-Variables Design Exploration

TL;DR

This paper tackles the need for explainability in surrogate-model-based engineering design, especially with mixed-variable inputs. It introduces SMT-EX, an explainability module added to SMT 2.0, which integrates SHAP, PDP, ICE, Sobol' indices, and split conformal prediction to uncover input–output relationships and quantify uncertainty. The authors validate SMT-EX on two engineering problems: a 10-variable continuous wing weight function and a 3-variable mixed-categorical cantilever beam, showing how global sensitivity, local explanations, and physically meaningful uncertainty intervals complement predictive performance. The results demonstrate that SMT-EX provides multiple, coherent perspectives for knowledge discovery and trust in surrogate-assisted design, with the tool available as open-source software.

Abstract

Surrogate models are of high interest for many engineering applications, serving as cheap-to-evaluate time-efficient approximations of black-box functions to help engineers and practitioners make decisions and understand complex systems. As such, the need for explainability methods is rising and many studies have been performed to facilitate knowledge discovery from surrogate models. To respond to these enquiries, this paper introduces SMT-EX, an enhancement of the open-source Python Surrogate Modeling Toolbox (SMT) that integrates explainability techniques into a state-of-the-art surrogate modelling framework. More precisely, SMT-EX includes three key explainability methods: Shapley Additive Explanations, Partial Dependence Plot, and Individual Conditional Expectations. A peculiar explainability dependency of SMT has been developed for such purpose that can be easily activated once the surrogate model is built, offering a user-friendly and efficient tool for swift insight extraction. The effectiveness of SMT-EX is showcased through two test cases. The first case is a 10-variable wing weight problem with purely continuous variables and the second one is a 3-variable mixed-categorical cantilever beam bending problem. Relying on SMT-EX analyses for these problems, we demonstrate its versatility in addressing a diverse range of problem characteristics. SMT-Explainability is freely available on Github: https://github.com/SMTorg/smt-explainability .

Figures (16)

• Figure 1: Partial dependence plots and individual conditional expectations for the wing weight problem.
• Figure 2: SHAP dependence plots for the wing weight problem.
• Figure 3: Feature importance of the wing weight problem based on PDP and SHAP.
• Figure 4: First and total order Sobol' indices for the wing weight problem.
• Figure 5: Prediction intervals for the wing weight problem.