A new wavelet-based variational family with copula dependence structures

Abstract

Variational inference (VI) has become a widely used approach for scalable Bayesian inference, but its performance strongly depends on the flexibility of the chosen variational family. In this work, we propose a novel variational family that combines wavelet-based representations for marginal posterior densities with copula functions to model dependence structures. The marginal distributions are constructed using coefficients from the discrete wavelet transform, providing a flexible and adaptive framework capable of capturing complex features such as asymmetry. The joint distribution is then obtained through a copula, allowing for explicit modeling of dependence among parameters, including both independence and Gaussian copula structures. We develop an efficient estimation procedure based on Monte Carlo approximations of the evidence lower bound (ELBO) and automatic differentiation, enabling scalable optimization using gradient-based methods. Through extensive simulation studies, including logistic regression, sparse linear models, and hierarchical models, we demonstrate that the proposed approach achieves posterior mean estimates comparable to Markov chain Monte Carlo (MCMC) methods, while providing improved uncertainty quantification relative to standard variational approaches. Applications to hierarchical logistic regression and Bayesian conditional transformation models further illustrate the practical advantages of the method in complex, high dimensional settings. The proposed wavelet copula variational family offers a flexible and computationally efficient alternative for Bayesian inference.

Figures (9)

• Figure 1: Posterior standard deviation (Std) from MCMC method (y) and VB method with mean-field Wavelet family (x) for the scenario with $n = 1000$ and $p = 10$ (label (a)) and with $n = 5000$ and $p = 50$ (label (b)).
• Figure 2: Posterior standard deviation (Std) from MCMC method (y) and VB method with Wavelet copula Gaussian family (x) for the scenario with $n = 1000$ and $p = 10$ (label (a)) and with $n = 5000$ and $p = 50$ (label (b)).
• Figure 3: Mean Absolute Error (MAE) of the model parameters across scenarios, (a), for Wavelet family and ADVI.
• Figure 4: Mean Absolute Error (MAE) in a test set across scenarios, from one to eight, for Wavelet family and ADVI.
• Figure 5: Scatter plot in (a) shows the comparison between MCMC estimates ($\hat{b}_{MCMC}$) and VB estimates $\hat{b}_{VB}$. Scatter plot in (b) shows the comparison between MAE of MCMC estimates ($\hat{b}_{MCMC}$) and MAE of VB estimates $\hat{b}_{VB}$. Both for scenario $\rho = 0$, $n_{ind} = 100, n_{rep} = 10$.