A Generalized Tangent Approximation based Variational Inference Framework for Strongly Super-Gaussian Likelihoods
Somjit Roy, Pritam Dey, Debdeep Pati, Bani K. Mallick
TL;DR
This work advances variational inference by generalizing tangent-approximation VI to strongly super-Gaussian likelihoods, enabling deterministic, geometry-driven updates with convergence guarantees. The TAVIE-SSG framework introduces alpha-fractional variational posteriors and an EM algorithm that scales linearly with data and is highly parallelizable. Theoretical results provide convergence and near-minimax variational-risk bounds under both fractional and standard likelihoods, while extensive experiments on heavy-tailed regression, Bayesian quantile regression, and spatial transcriptomics demonstrate superior scalability and competitive accuracy relative to state-of-the-art VI and MCMC methods. The approach offers a principled, scalable alternative for complex Bayesian models with non-Gaussian, heavy-tailed, or discrete responses, with practical implications across biostatistics, genomics, and econometrics.
Abstract
Variational inference, as an alternative to Markov chain Monte Carlo sampling, has played a transformative role in enabling scalable computation for complex Bayesian models. Nevertheless, existing approaches often depend on either rigid model-specific formulations or stochastic black-box optimization routines. Tangent approximation is a principled class of structured variational methods that exploits the geometry of the underlying probability model. However, its utility has largely been confined to logistic regression and related modeling regimes. In this article, we propose a novel variational framework based on tangent transformation for a broad class of probability models characterized by strongly super-Gaussian likelihoods. Our method leverages convex duality to construct tangent minorants of the log-likelihood, thereby inducing conjugacy with Gaussian priors over model parameters in an otherwise intractable setup. Under mild assumptions on the data-generating mechanism, we establish algorithmic convergence guarantees, a contribution that stands in contrast to the limited theoretical assurances typically available for black-box variational methods. Additionally, we derive near-minimax optimal bounds for the variational risk. Superior performance of our proposed methodology is illustrated on simulated and real-data scenarios that challenge state-of-the-art variational algorithms in terms of scalability and their ability to consistently capture complex underlying data structure.