Multi-level Monte Carlo Dropout for Efficient Uncertainty Quantification

TL;DR

This work develops a multilevel Monte Carlo framework for uncertainty quantification with MC-dropout in neural surrogates, treating dropout masks as a fidelity dimension controlled by the number of stochastic forward passes. By coupling coarse and fine dropout evaluations through shared masks, the authors create telescoping estimators for both the dropout-induced predictive mean and variance that remain unbiased while significantly reducing sampling variance at a fixed budget. They derive explicit bias, variance, and cost expressions, plus optimal cross-level allocation rules and ladder design guidance, and validate the approach on forward and inverse PINN Uzawa benchmarks, showing variance-rate predictions and fixed-cost advantages over single-fidelity MC-dropout. The results suggest substantial practical gains for calibrated uncertainty estimation in physics-informed surrogates, enabling accurate UQ within real-time or budget-constrained settings.

Abstract

We develop a multilevel Monte Carlo (MLMC) framework for uncertainty quantification with Monte Carlo dropout. Treating dropout masks as a source of epistemic randomness, we define a fidelity hierarchy by the number of stochastic forward passes used to estimate predictive moments. We construct coupled coarse--fine estimators by reusing dropout masks across fidelities, yielding telescoping MLMC estimators for both predictive means and predictive variances that remain unbiased for the corresponding dropout-induced quantities while reducing sampling variance at fixed evaluation budget. We derive explicit bias, variance and effective cost expressions, together with sample-allocation rules across levels. Numerical experiments on forward and inverse PINNs--Uzawa benchmarks confirm the predicted variance rates and demonstrate efficiency gains over single-level MC-dropout at matched cost.

Figures (11)

• Figure 1: Schematic of the surrogate network architecture in its deterministic form. The trained parameters $\hat{\theta}$ are fixed and repeated evaluations produce identical outputs $\mathscr{D}(x;\hat{\theta})$.
• Figure 2: The same network evaluated with dropout active. Each forward pass draws dropout masks (encoded in $\theta \sim \pi_{\hat{\theta}}$), yielding a randomised output $\mathscr{D}(x;\theta)$. Repeated evaluations resample $\theta$ and can therefore produce different outputs.
• Figure 3: Single-fidelity single-level MC-dropout estimators at increasing inner fidelity $T$. Each subplot shows the estimated mean $Y(x,T)$ and the confidence bands $Y(x,T)\pm \sqrt{V(x,T)}$ and $Y(x,T)\pm 2\sqrt{V(x,T)}$. The exact solution \ref{['eq:numerics:boundary_layer_exact_solution']} is shown as a dashed curve.
• Figure 4: Empirical estimator variances versus inner fidelity $T$ in the forward problem. Each subplot reports the discrete $L^1$ norm of the empirical outer-sample variance for the corresponding estimator, together with a log--log linear fit and a 99% confidence interval for the fitted slope.
• Figure 5: Dependence of empirical estimator variances on dropout probability $p_{\mathrm{drop}}$ for the forward problem at fixed $T_0=10^4$ and $M_0=10$. The dashed curves show the fitted model \ref{['eq:numerics:logit_model']} with 99% confidence bands.

Theorems & Definitions (14)

• Lemma 2.1: Moments of single-fidelity estimators
• Remark 2.2: Estimator hierarchy rather than discretisation hierarchy
• Lemma 2.3: Moments of the MLMC mean estimator
• Lemma 2.4: Covariance of unbiased sample-variance estimators with overlapping samples
• Theorem 2.5: Moments of the MLMC variance estimator
• proof
• Remark 2.6
• Lemma 3.1: Optimal allocation for the mean estimator
• Lemma 3.2: Optimal allocation for the variance estimator
• proof : Proof of Lemma \ref{['lem:mlmc:single_fidelity_moments']}