Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Bayesian Generative Modeling Approach for Arbitrary Conditional Inference

Qiao Liu, Wing Hung Wong

TL;DR

The paper tackles flexible conditional inference across arbitrary partitions of a high-dimensional vector $X$ by introducing Bayesian Generative Modeling (BGM), a latent-variable framework that learns a joint distribution $P_X(X)$ via a low-dimensional latent $Z$ and an iterative updating scheme. After training, BGM enables any conditional distribution $P(X_B|X_A)$ without retraining, by drawing $Z|X_A$ with HMC and sampling $X_B|Z,X_A$ from the closed-form Gaussian conditionals, yielding calibrated predictive intervals. The authors establish convergence of the stochastic updates, law-level consistency of the learned model, and conditional-risk bounds for arbitrary inference, and demonstrate superior point predictions and interval calibration against conformal baselines in simulations and MNIST imputation tasks. This train-once, infer-anywhere approach offers a scalable, uncertainty-aware engine for modern data analysis, with implications for missing data, multimodal prediction, and risk-sensitive decision making.

Abstract

Modern data analysis increasingly requires flexible conditional inference P(X_B | X_A) where (X_A, X_B) is an arbitrary partition of observed variable X. Existing conditional inference methods lack this flexibility as they are tied to a fixed conditioning structure and cannot perform new conditional inference once trained. To solve this, we propose a Bayesian generative modeling (BGM) approach for arbitrary conditional inference without retraining. BGM learns a generative model of X through an iterative Bayesian updating algorithm where model parameters and latent variables are updated until convergence. Once trained, any conditional distribution can be obtained without retraining. Empirically, BGM achieves superior prediction performance with well calibrated predictive intervals, demonstrating that a single learned model can serve as a universal engine for conditional prediction with uncertainty quantification. We provide theoretical guarantees for the convergence of the stochastic iterative algorithm, statistical consistency and conditional-risk bounds. The proposed BGM framework leverages the power of AI to capture complex relationships among variables while adhering to Bayesian principles, emerging as a promising framework for advancing various applications in modern data science. The code for BGM is freely available at https://github.com/liuq-lab/bayesgm.

A Bayesian Generative Modeling Approach for Arbitrary Conditional Inference

TL;DR

The paper tackles flexible conditional inference across arbitrary partitions of a high-dimensional vector by introducing Bayesian Generative Modeling (BGM), a latent-variable framework that learns a joint distribution via a low-dimensional latent and an iterative updating scheme. After training, BGM enables any conditional distribution without retraining, by drawing with HMC and sampling from the closed-form Gaussian conditionals, yielding calibrated predictive intervals. The authors establish convergence of the stochastic updates, law-level consistency of the learned model, and conditional-risk bounds for arbitrary inference, and demonstrate superior point predictions and interval calibration against conformal baselines in simulations and MNIST imputation tasks. This train-once, infer-anywhere approach offers a scalable, uncertainty-aware engine for modern data analysis, with implications for missing data, multimodal prediction, and risk-sensitive decision making.

Abstract

Modern data analysis increasingly requires flexible conditional inference P(X_B | X_A) where (X_A, X_B) is an arbitrary partition of observed variable X. Existing conditional inference methods lack this flexibility as they are tied to a fixed conditioning structure and cannot perform new conditional inference once trained. To solve this, we propose a Bayesian generative modeling (BGM) approach for arbitrary conditional inference without retraining. BGM learns a generative model of X through an iterative Bayesian updating algorithm where model parameters and latent variables are updated until convergence. Once trained, any conditional distribution can be obtained without retraining. Empirically, BGM achieves superior prediction performance with well calibrated predictive intervals, demonstrating that a single learned model can serve as a universal engine for conditional prediction with uncertainty quantification. We provide theoretical guarantees for the convergence of the stochastic iterative algorithm, statistical consistency and conditional-risk bounds. The proposed BGM framework leverages the power of AI to capture complex relationships among variables while adhering to Bayesian principles, emerging as a promising framework for advancing various applications in modern data science. The code for BGM is freely available at https://github.com/liuq-lab/bayesgm.
Paper Structure (18 sections, 4 theorems, 44 equations, 6 figures, 2 tables)

This paper contains 18 sections, 4 theorems, 44 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Problem Setup
  4. Generative Process of Observed Data
  5. Stochastic Iterative Updating Algorithm
  6. Arbitrary Conditional Inference
  7. Model Initialization
  8. Model Hyperparameters
  9. Theoretical Results
  10. Convergence of the stochastic iterative updating algorithm
  11. Statistical Consistency of the Learned Generative Model
  12. Conditional‑Risk Bounds for Arbitrary Conditional Inference
  13. Empirical Results
  14. Model Evaluation
  15. Baseline Methods
  16. ...and 3 more sections

Key Result

Theorem 1

(Convergence to stationary points) Every limit point $w_*$ of the sequence $\{w_t\}$ is almost surely first‑order stationary.

Figures (6)

  • Figure 1: The overview of BGM model. In the training stage, BGM serves as a generative model to learn the distribution of $P_X(X)$. Once fitted, BGM is capable of conditional inference under arbitrary partition of the observed variable $X$ in the inference stage. Variables are in rectangles and functions are in circles with incoming arrows indicating inputs to the function and outgoing arrows indicating outputs. The dotted arrow (green) represents Bayesian inference for the posterior of latent variable $Z$.
  • Figure 2: The comparison of the estimated prediction intervals from BGM the top three conformal prediction methods. Each dot represents a point in the held-out testing set. (a) $p=50$; (b)$p=100$; (c)$p=300$.
  • Figure 3: Data imputation experiments on MNIST dataset with BGM. (a) True images from held-out testing set. (b) Generated images from BGM after model training. (c) Testing images with random missing patterns (six random $5\times5$ squares as masks). (d) The imputed results by a trained BGM model given (c) as input. The posterior mean is used for estimating each missing pixel.
  • Figure 4: Data imputation with uncertainty qualification on MNIST dataset. (a) Imputation improves MNIST test classification accuracy by different methods. We varied the level of missingness (e.g., number of random $5\times5$ holes). (b-f) BGM imputed images and uncertainty heatmap with different missing patterns. (b) A $5 \times 13$ stripe mask in the middle. (c) A $5 \times 5$ square mask in the upper left. (d) A $5 \times 5$ square mask in the upper right (e) A $5 \times 5$ square mask in the lower left. (f) A $5 \times 5$ square mask in the lower right. Note that the pixels within the green rectangles are imputed by BGM posterior mean and the uncertainty is calculated by the average prediction interval length with $\alpha=0.05$ across all testing images.
  • Figure S1: More imputation results of BGM on MNIST testing set with different level of missingness. (a) Test images with different level of missingness. (b) Imputed images with a trained BGM model.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Theorem 1
  • Theorem 2
  • Theorem 3: Law-level Consistency
  • Theorem 4: Conditional Excess Risk Bound