Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multiproposal Elliptical Slice Sampling

Guillermina Senn, Nathan Glatt-Holtz, Giulia Carigi, Andrew Holbrook, Håkon Tjelmeland

TL;DR

Multiproposal Elliptical Slice Sampling is introduced, a self-tuning multiproposal Markov chain Monte Carlo method for Bayesian inference with Gaussian priors that is particularly well suited for Bayesian PDE inverse problems.

Abstract

We introduce Multiproposal Elliptical Slice Sampling, a self-tuning multiproposal Markov chain Monte Carlo method for Bayesian inference with Gaussian priors. Our method generalizes the Elliptical Slice Sampling algorithm by 1) allowing multiple candidate proposals to be sampled in parallel at each self-tuning step, and 2) basing the acceptance step on a distance-informed transition matrix that can favor proposals far from the current state. This allows larger moves in state space and faster self-tuning, at essentially no additional wall clock time for expensive likelihoods, and results in improved mixing. We additionally provide theoretical arguments and experimental results suggesting dimension-robust mixing behavior, making the algorithm particularly well suited for Bayesian PDE inverse problems.

Multiproposal Elliptical Slice Sampling

TL;DR

Multiproposal Elliptical Slice Sampling is introduced, a self-tuning multiproposal Markov chain Monte Carlo method for Bayesian inference with Gaussian priors that is particularly well suited for Bayesian PDE inverse problems.

Abstract

We introduce Multiproposal Elliptical Slice Sampling, a self-tuning multiproposal Markov chain Monte Carlo method for Bayesian inference with Gaussian priors. Our method generalizes the Elliptical Slice Sampling algorithm by 1) allowing multiple candidate proposals to be sampled in parallel at each self-tuning step, and 2) basing the acceptance step on a distance-informed transition matrix that can favor proposals far from the current state. This allows larger moves in state space and faster self-tuning, at essentially no additional wall clock time for expensive likelihoods, and results in improved mixing. We additionally provide theoretical arguments and experimental results suggesting dimension-robust mixing behavior, making the algorithm particularly well suited for Bayesian PDE inverse problems.
Paper Structure (16 sections, 3 theorems, 56 equations, 9 figures, 2 tables, 2 algorithms)

This paper contains 16 sections, 3 theorems, 56 equations, 9 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Multiproposal Elliptical Slice Sampling
  3. The updating procedure
  4. Validity of the updating procedure
  5. Transition Matrix
  6. Computational and statistical scaling
  7. Related parallel-ESS work
  8. Results
  9. Gaussian process classification
  10. Blind deconvolution
  11. Toy model for solute transport
  12. Concluding remarks
  13. Proof of Lemma \ref{['lem:equivalence_indicators']}
  14. Proof of Lemma \ref{['lem:recursive_mess']}
  15. Proof of proposition \ref{['thm:stationary_mess']}
  16. ...and 1 more sections

Key Result

Proposition 2.1

The Multiproposal Elliptical Slice Sampling updating procedure leaves the target distribution $\pi(\bm{x})$ invariant.

Figures (9)

  • Figure 1: Ellipses at different iterations of a MESS ($M=5$) chain for posterior estimation of the non-parametric example in Section \ref{['sec:results']}. (a) The slice is discontinuous, and one of five proposals lies on a segment excluding the current state. (b, c) None of the first five angles is valid, shrinking the interval; four subsequent proposals are valid, and one is accepted.
  • Figure 2: Average number of shrinking steps and likelihood evaluations per iteration, as a function of the number of proposals, for the Gaussian process classification example.
  • Figure 3: Posterior samples and selected traceplots for the marginal blur estimation in the blind deconvolution model. MESS ($M=20$, angular) explores both modes in 4% of the time used by HMC.
  • Figure 4: Posterior marginal histograms for $a_{01}, a_{02}$, and $a_{12}$ on the diagonal, and corresponding pairwise density plots off-diagonal, for the solute transport model at $d=10$, computed with 300k samples from MESS $(M=100)$. The highly non-Gaussian marginal posteriors and complex correlation structures are a result of the non-linear forward map.
  • Figure 5: Posterior traceplots (30k iterations) and marginal histogram (300k iterations) for $a_{01}$ in the solute transport model, at $d=10$. All chains satisfactorily explore the posterior although with different exploration behaviour.
  • ...and 4 more figures

Theorems & Definitions (11)

  • Definition 2.1: MESS Shrinking rule
  • Definition 2.2: Transition matrix
  • Remark 2.1: Invariance
  • Proposition 2.1: Invariance
  • Remark 2.2
  • Lemma 2.1: Invariance of support
  • Lemma 2.2: Ordering and flipping properties of intervals
  • proof : Proof of Lemma \ref{['lem:equivalence_indicators']}
  • proof : Proof for \ref{['eq:rest1']}$(a)-(d) \iff$\ref{['eq:rest2']}$(i-ii)$
  • proof : Proof for (\ref{['eq:rest1']})$(e)-(h) \iff$ (\ref{['eq:rest2']})$(iii-vi)$
  • ...and 1 more