Neural variational inference for cutting feedback during uncertainty propagation

TL;DR

The paper addresses the practical problem of propagating upstream uncertainty into downstream Bayesian analyses without allowing downstream data to influence upstream parameters. It introduces NeVI-Cut, a modular neural variational method that uses upstream posterior samples and conditional normalizing flows to learn the downstream conditional posterior $q(\theta|\eta)$, yielding the cut-posterior through $p_{cut}(\theta,\eta|D_1,D_2)=q(\theta|\eta)\,p(\eta|D_1)$ in a fixed-data regime. The authors provide theoretical guarantees, including universal KL approximation rates for conditional flows and convergence of the NeVI-Cut estimator, with explicit rates for mainstream flow classes. Through simulation and two real-data applications, NeVI-Cut demonstrates substantial computational gains over nested MCMC and superior accuracy to parametric variational cuts, while preserving modularity and privacy by not requiring upstream data. The work advances cutting-feedback methods by combining flexible, flow-based variational families with a principled fixed-data theory, and it offers practical, scalable implementations with public code.

Abstract

In many scientific applications, uncertainty of estimates from an earlier (upstream) analysis needs to be propagated in subsequent (downstream) Bayesian analysis, without feedback. Cutting feedback methods, also termed cut-Bayes, achieve this by constructing a cut-posterior distribution that prevents backward information flow. Cutting feedback like nested MCMC is computationally challenging while variational inference (VI) cut-Bayes methods need two variational approximations and require access to the upstream data and model. In this manuscript we propose, NeVI-Cut, a provably accurate and modular neural network-based variational inference method for cutting feedback. We directly utilize samples from the upstream analysis without requiring access to the upstream data or model. This simultaneously preserves modularity of analysis and reduces approximation errors by avoiding a variational approximation for the upstream model. We then use normalizing flows to specify the conditional variational family for the downstream parameters and estimate the conditional cut-posterior as a variational solution of Monte Carlo average loss over all the upstream samples. We provide theoretical guarantees on the NeVI-Cut estimate to approximate any cut-posterior. Our results are in a fixed-data regime and provide convergence rates of the actual variational solution, quantifying how richness of the neural architecture and the complexity of the target cut-posterior dictate the approximation quality. In the process, we establish new results on uniform Kullback-Leibler approximation rates of conditional normalizing flows. Simulation studies and two real-world analyses illustrate how NeVI-Cut achieves significant computational gains over traditional cutting feedback methods and is considerably more accurate than parametric variational cut approaches.

Figures (6)

• Figure 1: Comparison of NeVI-Cut estimates using RQ-NSF(AR) and UMNN flows with the true distributions when estimating the conditional cut-posterior $p(\theta \,|\, \eta=\eta_0)$ (left) and marginal cut-posterior $p(\theta)$ (right).
• Figure 2: Posterior distributions of calibrated cause-specific mortality fractions (CSMFs) in neonates (0-27 days) based on VA-only data collected by the Countrywide Mortality Surveillance for Action (COMSA) program in Mozambique. NeVI-Cut effectively captures complex distributional features, including multimodality and skewness, achieving similar performance to nested MCMC. Con. mal., Sep./Menin./Inf., IPRE denote congenital malformation, sepsis/meningitis/infection, and intrapartum-related events.
• Figure 3: Left panel: Quadratic Wasserstein distances in centered log-ratio space between each method and nested MCMC across algorithms. Right panel: Runtime comparison of NeVI-Cut vs nested MCMC.
• Figure S1: Paradigm of the cutting feedback from downstream to upstream. Black arrows indicate parameters specifying a model, blue dotted arrows indicate the direction of information flow from a data source to parameters.
• Figure S2: Posterior distributions for the downstream model parameters in the HPV data analysis. The left panel shows joint posteriors of $(\eta_1, \eta_2)$ under full Bayes and different cut-Bayes methods. The right panel showcases marginal posteriors of $(\eta_1, \eta_2)$ using different cut-Bayes methods compared with multiple imputation-based nested MCMC.

Theorems & Definitions (16)

• Lemma 1
• Lemma 2
• Theorem 1: Uniform KL rates of conditional flows
• Theorem 2: Rate for Conditional Unconstrained Monotonic Neural Networks
• Theorem 3: Rate for Conditional Rational Quadratic Neural Spline Flows
• Corollary 1: Integral-based NeVI-Cut
• Theorem 4: Sample-based NeVI-Cut
• proof : Proof of Lemma \ref{['lem:quantile']}
• proof : Proof of Lemma \ref{['lem:lips']}
• proof : Proof of Theorem \ref{['thm:ckl-explicit']}