Quasi-Bayes meets Vines

TL;DR

The paper targets scalable, explicit-density estimation in high dimensions by marrying Quasi-Bayesian predictive densities with Sklar's theorem. It decomposes the joint predictive $oldsymbol{p}^{(n)}(oldsymbol{x})$ into univariate marginal predictives, estimated by fast QB recursions, and a flexible vine copula to capture dependence, yielding an analytical, copula-based density with a convergence rate that can be dimension-independent under simplified vine assumptions. The authors introduce the QB-Vine framework, provide convergence results for marginals and copulas, and demonstrate robustness through energy-score-based hyperparameter tuning and parallelizable computation. Empirically, QB-Vine achieves state-of-the-art or competitive density estimation and supervised task performance on moderate-to-high dimensional data (up to $d\sim 64$) with relatively small training samples, highlighting its data efficiency and scalability compared to neural and other Bayesian approaches.

Abstract

Recently proposed quasi-Bayesian (QB) methods initiated a new era in Bayesian computation by directly constructing the Bayesian predictive distribution through recursion, removing the need for expensive computations involved in sampling the Bayesian posterior distribution. This has proved to be data-efficient for univariate predictions, but extensions to multiple dimensions rely on a conditional decomposition resulting from predefined assumptions on the kernel of the Dirichlet Process Mixture Model, which is the implicit nonparametric model used. Here, we propose a different way to extend Quasi-Bayesian prediction to high dimensions through the use of Sklar's theorem by decomposing the predictive distribution into one-dimensional predictive marginals and a high-dimensional copula. Thus, we use the efficient recursive QB construction for the one-dimensional marginals and model the dependence using highly expressive vine copulas. Further, we tune hyperparameters using robust divergences (eg. energy score) and show that our proposed Quasi-Bayesian Vine (QB-Vine) is a fully non-parametric density estimator with \emph{an analytical form} and convergence rate independent of the dimension of data in some situations. Our experiments illustrate that the QB-Vine is appropriate for high dimensional distributions ($\sim$64), needs very few samples to train ($\sim$200) and outperforms state-of-the-art methods with analytical forms for density estimation and supervised tasks by a considerable margin.

Figures (2)

• Figure 1: Overview of the Quasi-Bayesian Vine model. The joint predictive density $\mathbf{p}^{(n)}\left(x_1,\ldots, x_d\right)$ is modelled nonparametrically through a copula decomposition into univariate marginal predictives and a multivariate copula. Left: Univariate predictive densities $p_i^{(n)}$ are modelled with a data-efficient Quasi-Bayesian recursion using observed samples $\{x_i^{(k)}\}_{k=1}^n$ alone, bypassing posterior integration steps. Right: A vine copula model decomposes the high-dimensional copula into $\frac{d(d-1)}{2}$ two-dimensional copulas nonparametrically estimated with kernel density copula estimators represented by a graphical structure between dimensions
• Figure 2: Density estimation on the Digits data ($n=1797,d=64$) with reduced training sizes for the QB-Vine against other models fitted on the full training set. The QB-Vine achieves competitive performance for training sizes as little as $n=50$ and outperforms all competitors once $n>200$.

Theorems & Definitions (8)

• Theorem 3.1
• Lemma 3.2
• proof
• Theorem 3.3
• proof
• Theorem A.1: Sklar sklar
• Definition A.2: Regular Vine Copulas
• proof