Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach

TL;DR

This paper addresses Bayesian experimental design (BED) under model discrepancy by introducing a hybrid framework that combines traditional BED for low-dimensional physical parameters with an auto-differentiable ensemble Kalman inversion (AD-EKI) to efficiently calibrate high-dimensional discrepancy terms. By deriving differentiable estimates of KL divergence and expected information gain through AD-EKI, the approach enables gradient-based design optimization even in the presence of nonlinear, high-dimensional discrepancy models represented by neural networks. The authors demonstrate the method on a convection–diffusion contaminant-source problem, showing that informative designs identified via AD-EKI lead to robust parameter inference for both the physics and the discrepancy, with favorable scalability in ensemble size and iteration count. The work contributes a practical, scalable solution to BED with model discrepancy and suggests broad applicability to bilevel optimization and differentiable programming challenges in related fields.

Abstract

Bayesian experimental design (BED) offers a principled framework for optimizing data acquisition by leveraging probabilistic inference. However, practical implementations of BED are often compromised by model discrepancy, i.e., the mismatch between predictive models and true physical systems, which can potentially lead to biased parameter estimates. While data-driven approaches have been recently explored to characterize the model discrepancy, the resulting high-dimensional parameter space poses severe challenges for both Bayesian updating and design optimization. In this work, we propose a hybrid BED framework enabled by auto-differentiable ensemble Kalman inversion (AD-EKI) that addresses these challenges by providing a computationally efficient, gradient-free alternative to estimate the information gain for high-dimensional network parameters. The AD-EKI allows a differentiable evaluation of the utility function in BED and thus facilitates the use of standard gradient-based methods for design optimization. In the proposed hybrid framework, we iteratively optimize experimental designs, decoupling the inference of low-dimensional physical parameters handled by standard BED methods, from the high-dimensional model discrepancy handled by AD-EKI. The identified optimal designs for the model discrepancy enable us to systematically collect informative data for its calibration. The performance of the proposed method is studied by a classical convection-diffusion BED example, and the hybrid framework enabled by AD-EKI efficiently identifies informative data to calibrate the model discrepancy and robustly infers the unknown physical parameters in the modeled system. Besides addressing the challenges of BED with model discrepancy, AD-EKI also potentially fosters efficient and scalable frameworks in many other areas with bilevel optimization, such as meta-learning and structure optimization.

Figures (13)

• Figure 1: Schematic diagram of the hybrid BED framework: iterative learning of the parameters of a physics-based model and the model discrepancy.
• Figure 2: Concentration value at different times in a convection-diffusion field. The numerical simulation is performed in a larger domain ($[-2,3]^2$) and presented in a smaller one ($[0,1]^2$) to emphasize the areas close to the source location.
• Figure 3: Posterior results of physical parameter $\{\theta_x,\theta_y\}$ via the hybrid approach on parametric error. The $\{\theta_x,\theta_y\}$ space is $[0,1]^2$ as before, with a zoomed-in view $[0.2,0.8]\times[0.0,0.6]$ to highlight detailed behaviors.
• Figure 4: Results of the parametric error case: panel (a) shows a quantitative analysis of the posterior distribution of physical parameters, panel (b) shows the updating process of error parameter.
• Figure 5: Design optimization results of the parametric error case: panel (a) shows the trajectory of design optimization via ensemble-based utility function and panel (b) shows the information gain at each EKI iteration of different designs. The gray dots in panel (a) represent the sequentially updated designs during the EKI iterations, with darker dots indicating earlier iterations and lighter dots indicating more recent iterations.