Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analysis of two-component Gibbs samplers using the theory of two projections

Qian Qin

TL;DR

It is found that in terms of asymptotic variance, the two-component random-scan Gibbs sampler is never much worse, and could be considerably better than its deterministic-scan counterpart, provided that the selection probability is appropriately chosen.

Abstract

The theory of two projections is utilized to study two-component Gibbs samplers. Through this theory, previously intractable problems regarding the asymptotic variances of two-component Gibbs samplers are reduced to elementary matrix algebra exercises. It is found that in terms of asymptotic variance, the two-component random-scan Gibbs sampler is never much worse, and could be considerably better than its deterministic-scan counterpart, provided that the selection probability is appropriately chosen. This is especially the case when there is a large discrepancy in computation cost between the two components. The result contrasts with the known fact that the deterministic-scan version has a faster convergence rate, which can also be derived from the method herein. On the other hand, a modified version of the deterministic-scan sampler that accounts for computation cost can outperform the random-scan version.

Analysis of two-component Gibbs samplers using the theory of two projections

TL;DR

It is found that in terms of asymptotic variance, the two-component random-scan Gibbs sampler is never much worse, and could be considerably better than its deterministic-scan counterpart, provided that the selection probability is appropriately chosen.

Abstract

The theory of two projections is utilized to study two-component Gibbs samplers. Through this theory, previously intractable problems regarding the asymptotic variances of two-component Gibbs samplers are reduced to elementary matrix algebra exercises. It is found that in terms of asymptotic variance, the two-component random-scan Gibbs sampler is never much worse, and could be considerably better than its deterministic-scan counterpart, provided that the selection probability is appropriately chosen. This is especially the case when there is a large discrepancy in computation cost between the two components. The result contrasts with the known fact that the deterministic-scan version has a faster convergence rate, which can also be derived from the method herein. On the other hand, a modified version of the deterministic-scan sampler that accounts for computation cost can outperform the random-scan version.
Paper Structure (17 sections, 23 theorems, 176 equations, 2 figures, 4 algorithms)

This paper contains 17 sections, 23 theorems, 176 equations, 2 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Basic properties of two-component Gibbs samplers
  4. Markov operators
  5. Halmos's Theory of Two Projections
  6. Comparing Asymptotic Variances
  7. Characterizing $V_{\scriptsize\hbox{D}}(f)$ and $V_{\scriptsize\hbox{R}}(f,r)$
  8. The comparison
  9. Data augmentation
  10. A Modified DG Sampler
  11. Convergence Rate
  12. Discussion
  13. Technical Results
  14. Proof of Proposition \ref{['pro:rate']}
  15. Supplement
  16. ...and 2 more sections

Key Result

Lemma 2.1

halmos1969two Assume that $M_{\scriptsize\hbox{R}} \neq \{0\}$. Then $P_1M_{\scriptsize\hbox{R}}$ and $(I-P_1)M_{\scriptsize\hbox{R}}$ are both nontrivial and have the same dimension. Moreover, there exist a unitary transformation $W: (I-P_1)M_{\scriptsize\hbox{R}} \to P_1 M_{\scriptsize\hbox{R}}$ a

Figures (2)

  • Figure 1: Graph of $r \mapsto k_i(r)$ for $i=1,2$.
  • Figure 2: Relationship between $\tau$ and $r$ as given in \ref{['eq:r']}. $\tau$ is given in log scale.

Theorems & Definitions (46)

  • Remark 1.1
  • Lemma 2.1
  • Remark 2.2
  • Remark 2.3
  • Lemma 2.4
  • Example 2.5
  • Proposition 3.1
  • proof
  • Proposition 3.2
  • proof
  • ...and 36 more