Surrogate Modeling and Explainable Artificial Intelligence for Complex Systems: A Workflow for Automated Simulation Exploration

TL;DR

This paper presents a unified workflow that integrates surrogate modeling with Explainable AI (XAI) to enable fast, transparent exploration of complex-system simulations. By training lightweight emulators on strategically designed experiments, the approach provides fast predictions and rigorous uncertainty quantification while supporting global and local explanations. The authors demonstrate the workflow on two distinct case studies—the DRAGON hybrid-electric aircraft design and a Schelling urban-segregation ABM—showing how surrogate-XAI coupling reveals nonlinear interactions, emergent behaviors, and actionable levers, all while signaling when surrogates require more data or alternative architectures. Overall, the framework advances scalable, trustworthy automated simulation exploration for design and policy analysis, with practical guidance on surrogate selection, interpretation, and validation across heterogeneous problem domains.

Abstract

Complex systems are increasingly explored through simulation-driven engineering workflows that combine physics-based and empirical models with optimization and analytics. Despite their power, these workflows face two central obstacles: (1) high computational cost, since accurate exploration requires many expensive simulator runs; and (2) limited transparency and reliability when decisions rely on opaque blackbox components. We propose a workflow that addresses both challenges by training lightweight emulators on compact designs of experiments that (i) provide fast, low-latency approximations of expensive simulators, (ii) enable rigorous uncertainty quantification, and (iii) are adapted for global and local Explainable Artificial Intelligence (XAI) analyses. This workflow unifies every simulation-based complex-system analysis tool, ranging from engineering design to agent-based models for socio-environmental understanding. In this paper, we proposea comparative methodology and practical recommendations for using surrogate-based explainability tools within the proposed workflow. The methodology supports continuous and categorical inputs, combines global-effect and uncertainty analyses with local attribution, and evaluates the consistency of explanations across surrogate models, thereby diagnosing surrogate adequacy and guiding further data collection or model refinement. We demonstrate the approach on two contrasting case studies: a multidisciplinary design analysis of a hybrid-electric aircraft and an agent-based model of urban segregation. Results show that the surrogate model and XAI coupling enables large-scale exploration in seconds, uncovers nonlinear interactions and emergent behaviors, identifies key design and policy levers, and signals regions where surrogates require more data or alternative architectures.

Figures (19)

• Figure 1: Workflow for interpretable surrogate modeling in complex system co-design.
• Figure 2: Workflow for interpretable surrogate models for a design-oriented simulation.
• Figure 3: DRAGON aircraft mock-up ridel2021dragon.
• Figure 4: GSA results from partial dependence functions for the DRAGON problem.
• Figure 5: GSA results from the averaged SHAP values for the DRAGON problem.