Hierarchical random measures without tables

TL;DR

The paper tackles computational bottlenecks in hierarchical Bayesian nonparametrics by eliminating latent tables and modeling with normalized hierarchical completely random vectors (hCRVs) under a gamma hyperprior on the HDP concentration parameter. It develops a general posterior representation for normalized hCRVs, proving quasi-conjugacy and enabling exact or fast approximate sampling without table bookkeeping. Specializing to gamma-gamma hCRVs, the authors derive complete sampling algorithms (including exact gamma-based schemes) and demonstrate substantial efficiency gains and robust posterior behavior relative to the standard HDP, supported by simulation. The work further provides a flexible framework for constructing and eliciting dependent nonparametric priors with tractable multivariate increment structure, and shows how to calibrate dependence to achieve desired borrowing and shrinkage in multigroup settings.

Abstract

The hierarchical Dirichlet process is the cornerstone of Bayesian nonparametric multilevel models. Its generative model can be described through a set of latent variables, commonly referred to as tables within the popular restaurant franchise metaphor. The latent tables simplify the expression of the posterior and allow for the implementation of a Gibbs sampling algorithm to approximately draw samples from it. However, managing their assignments can become computationally expensive, especially as the size of the dataset and of the number of levels increase. In this work, we identify a prior for the concentration parameter of the hierarchical Dirichlet process that (i) induces a quasi-conjugate posterior distribution, and (ii) removes the need of tables, bringing to more interpretable expressions for the posterior, with both a faster and an exact algorithm to sample from it. Remarkably, this construction extends beyond the Dirichlet process, leading to a new framework for defining normalized hierarchical random measures and a new class of algorithms to sample from their posteriors. The key analytical tool is the independence of multivariate increments, that is, their representation as completely random vectors.

Figures (14)

• Figure 1: Conditional dependencies between random variables in Algorithms \ref{['alg:mcmc']} and \ref{['alg:exact']}. Red circles represent the computational bottlenecks; quantities in empty circles are sampled from gamma distributions. Model parameters and observations enter the sampling scheme at different steps. For simplicity, variables are reported up to scaling w.r.t. model parameters, e.g. $T$ instead of $\alpha T$.
• Figure 2: (Top) CPU time per effective sample for different algorithms with (a) growing number of groups $d$ and (b) growing number of observations per group $n_{i}$. (Bottom) Number of clusters for each experimental setting.
• Figure 3: CPU time per effective sample for the different algorithms, with growing number of clusters $k$. The number of groups $d = 5$ and observations per group $n_{i} = 12$ are fixed.
• Figure 4: Values of $\alpha$ and $\alpha_0$ for the gamma-gamma hCRV corresponding to fixed values of variance $\sigma^2$ and correlation $\rho$, when the set $A$ satisfies $P_0(A) = 1/2$.
• Figure 5: Posterior expected random means of the gamma-gamma hCRV model, for three groups of independent Poisson observations with means equal to $2$, $3$ and $4$, each of size $n_{i}=10$. Top: each plot has fixed variance parameter $\sigma^2$ and increasing correlation $\rho$. Bottom: each plot has fixed correlation and increasing variance. The prior mean $P_0 = N(10,1)$ is fixed for all plots. Posterior samples are obtained using the exact algorithm. The horizontal lines denote the empirical means.

Theorems & Definitions (35)

• Definition 1
• Definition 2
• Theorem 1
• Lemma 1
• Theorem 2
• Example 1
• Example 2
• Lemma 2
• Remark 1
• Proposition 1