Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IID Sampling from Intractable Distributions

Sourabh Bhattacharya

TL;DR

This work proposes a novel methodology for drawing iid realizations from any target distribution on the Euclidean space with arbitrary dimension to construct an appropriate infinite sequence of concentric closed ellipsoids, represent the target distribution as an infinite mixture on the centralEllipsoid and the ellip soidal annuli, and to construct efficient perfect samplers for the mixture components.

Abstract

We propose a novel methodology for drawing iid realizations from any target distribution on the Euclidean space with arbitrary dimension. No assumption of compact support is necessary for the validity of our theory and method. Our idea is to construct an appropriate infinite sequence of concentric closed ellipsoids, represent the target distribution as an infinite mixture on the central ellipsoid and the ellipsoidal annuli, and to construct efficient perfect samplers for the mixture components. In contrast with most of the existing works on perfect sampling, ours is not only a theoretically valid method, it is practically applicable to all target distributions on any dimensional Euclidean space and very much amenable to parallel computation. We validate the practicality and usefulness of our methodology by generating 10000 iid realizations from the standard distributions such as normal, Student's t with 5 degrees of freedom and Cauchy, for dimensions d = 1, 5, 10, 50, 100, as well as from a 50-dimensional mixture normal distribution. The implementation time in all the cases are very reasonable, and often less than a minute in our parallel implementation. The results turned out to be highly accurate. We also apply our method to draw 10000 iid realizations from the posterior distributions associated with the well-known Challenger data, a Salmonella data and the 160-dimensional challenging spatial example of the radionuclide count data on Rongelap Island. Again, we are able to obtain quite encouraging results with very reasonable computing time.

IID Sampling from Intractable Distributions

TL;DR

This work proposes a novel methodology for drawing iid realizations from any target distribution on the Euclidean space with arbitrary dimension to construct an appropriate infinite sequence of concentric closed ellipsoids, represent the target distribution as an infinite mixture on the centralEllipsoid and the ellip soidal annuli, and to construct efficient perfect samplers for the mixture components.

Abstract

We propose a novel methodology for drawing iid realizations from any target distribution on the Euclidean space with arbitrary dimension. No assumption of compact support is necessary for the validity of our theory and method. Our idea is to construct an appropriate infinite sequence of concentric closed ellipsoids, represent the target distribution as an infinite mixture on the central ellipsoid and the ellipsoidal annuli, and to construct efficient perfect samplers for the mixture components. In contrast with most of the existing works on perfect sampling, ours is not only a theoretically valid method, it is practically applicable to all target distributions on any dimensional Euclidean space and very much amenable to parallel computation. We validate the practicality and usefulness of our methodology by generating 10000 iid realizations from the standard distributions such as normal, Student's t with 5 degrees of freedom and Cauchy, for dimensions d = 1, 5, 10, 50, 100, as well as from a 50-dimensional mixture normal distribution. The implementation time in all the cases are very reasonable, and often less than a minute in our parallel implementation. The results turned out to be highly accurate. We also apply our method to draw 10000 iid realizations from the posterior distributions associated with the well-known Challenger data, a Salmonella data and the 160-dimensional challenging spatial example of the radionuclide count data on Rongelap Island. Again, we are able to obtain quite encouraging results with very reasonable computing time.

Paper Structure

This paper contains 25 sections, 40 equations, 15 figures.

Table of Contents

  1. Introduction
  2. The $iid$ sampling idea
  3. Choice of the sets $\boldsymbol{A}_i$
  4. Choice of $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$
  5. Dealing with the mixing probabilities $\pi(\boldsymbol{A}_i)$
  6. Uniform sampling from $\boldsymbol{A}_i$
  7. Lebesgue measure of $\boldsymbol{A}_i$
  8. A rejection sampling argument for validation of $\hat{\pi}(\boldsymbol{\theta})$
  9. Perfect sampling from $\hat{\pi}(\boldsymbol{\theta})$
  10. Minorization for $\pi_i$
  11. Split chain
  12. Perfect simulation from $\pi_i$
  13. The complete algorithm for $iid$ sample generation from $\pi$
  14. Experiments with standard distributions
  15. Simulation from normals
  16. ...and 10 more sections

Figures (15)

  • Figure 1: Schematic diagram: the conceptual overview.
  • Figure 2: Schematic diagram: perfect sampling.
  • Figure 3: Schematic diagram: parallel implementation.
  • Figure 4: Simulation from $100$-dimensional normal distribution. The red and blue colours denote the $iid$ sample based density and the true density, respectively.
  • Figure 5: Simulation from $100$-dimensional normal distribution. True and $iid$-based correlation structures for the first $20$ co-ordinates.
  • ...and 10 more figures