Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Bayes and Truncation approximations for Enriched Dirichlet process mixtures

Somnath Bhadra, Michael J. Daniels

Abstract

A common impediment in conducting inference for Bayesian nonparametric models is either the need for complex MCMC algorithms and/or computational run-time for large datasets. We propose solutions here for Enriched Dirichlet process mixtures (EDPM). We derive a variational Bayes estimator based on a previously developed truncation approximation for EDPMs. The variational Bayes estimator can be used in two ways: 1) to develop a more efficient truncation approximation; 2) as good initial values for a blocked Gibbs sampler based on this more efficient truncation approximation or for a polya urn sampler. We derive the accuracy of this more efficient truncation approximation and demonstrate how this allows for simple implementation of a blocked Gibbs Sampler EDPMs in Nimble. We confirm the validity of the approximations by simulations and illustrate on a real data set.

Variational Bayes and Truncation approximations for Enriched Dirichlet process mixtures

Abstract

A common impediment in conducting inference for Bayesian nonparametric models is either the need for complex MCMC algorithms and/or computational run-time for large datasets. We propose solutions here for Enriched Dirichlet process mixtures (EDPM). We derive a variational Bayes estimator based on a previously developed truncation approximation for EDPMs. The variational Bayes estimator can be used in two ways: 1) to develop a more efficient truncation approximation; 2) as good initial values for a blocked Gibbs sampler based on this more efficient truncation approximation or for a polya urn sampler. We derive the accuracy of this more efficient truncation approximation and demonstrate how this allows for simple implementation of a blocked Gibbs Sampler EDPMs in Nimble. We confirm the validity of the approximations by simulations and illustrate on a real data set.
Paper Structure (15 sections, 3 theorems, 58 equations, 3 tables)

This paper contains 15 sections, 3 theorems, 58 equations, 3 tables.

Table of Contents

  1. Introduction
  2. EDPM and a truncation approximation
  3. EDPM
  4. Truncation in EDPM
  5. Variational Bayes
  6. VB for truncated EDPM
  7. Truncation Approximation
  8. Truncation
  9. How to choose $N$ and $M_k$
  10. Comparison with fixed cluster scenario
  11. Blocked Gibbs Sampler
  12. Simulation to assess the truncation approximation
  13. Scenario I, Mixture model
  14. Scenario II, Non-mixture model
  15. Posterior distributions for Block Gibbs Sampler

Key Result

Theorem 4.1

For $M_k\to\infty\space\forall \space1\leq k\leq N,$ and $N\to \infty$

Theorems & Definitions (8)

  • Theorem 4.1
  • proof
  • Theorem 4.2
  • proof
  • Lemma 4.1
  • proof
  • Remark 4.1
  • Remark 5.1