Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Covariance estimation using Markov chain Monte Carlo

Yunbum Kook, Matthew S. Zhang

TL;DR

This work investigates the complexity of covariance matrix estimation for Gibbs distributions based on dependent samples from a Markov chain and shows that when $\pi$ satisfies a Poincar\'e inequality and the chain possesses a spectral gap, it can achieve similar sample complexity using MCMC as compared to an estimator constructed using i.d. samples.

Abstract

We investigate the complexity of covariance matrix estimation for Gibbs distributions based on dependent samples from a Markov chain. We show that when $π$ satisfies a Poincaré inequality and the chain possesses a spectral gap, we can achieve similar sample complexity using MCMC as compared to an estimator constructed using i.i.d. samples, with potentially much better query complexity. As an application of our methods, we show improvements for the query complexity in both constrained and unconstrained settings for concrete instances of MCMC. In particular, we provide guarantees regarding isotropic rounding procedures for sampling uniformly on convex bodies.

Covariance estimation using Markov chain Monte Carlo

TL;DR

This work investigates the complexity of covariance matrix estimation for Gibbs distributions based on dependent samples from a Markov chain and shows that when satisfies a Poincar\'e inequality and the chain possesses a spectral gap, it can achieve similar sample complexity using MCMC as compared to an estimator constructed using i.d. samples.

Abstract

We investigate the complexity of covariance matrix estimation for Gibbs distributions based on dependent samples from a Markov chain. We show that when satisfies a Poincaré inequality and the chain possesses a spectral gap, we can achieve similar sample complexity using MCMC as compared to an estimator constructed using i.i.d. samples, with potentially much better query complexity. As an application of our methods, we show improvements for the query complexity in both constrained and unconstrained settings for concrete instances of MCMC. In particular, we provide guarantees regarding isotropic rounding procedures for sampling uniformly on convex bodies.

Paper Structure

This paper contains 45 sections, 27 theorems, 112 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Results
  3. Covariance estimation via MCMC.
  4. Applications.
  5. Related work
  6. Covariance estimation.
  7. Concentration inequalities under dependence.
  8. Sampling under functional inequalities.
  9. Organization
  10. Preliminaries
  11. Notation
  12. Background
  13. Covariance estimation using dependent samples
  14. Main results
  15. Proof sketch
  16. ...and 30 more sections

Key Result

Theorem 1

Assume that $\pi$ satisfies a Poincaré inequality and $P$ has a spectral gap. For $\varepsilon>0$ and $\delta\in (0, d)$, the estimator $\overline \Sigma \coloneqq \frac{1}{N} \sum_{i=1}^N (X_i - \overline X)^{\otimes 2}$ satisfies with high probability that $|\overline{\Sigma}-\textup{Cov}(\pi)|\pr

Theorems & Definitions (57)

  • Theorem 1: Informal; see Theorem \ref{['thm:add-cov-Poincare']}
  • Definition 2
  • Definition 3
  • Definition 4: Probability divergences
  • Theorem 6: neeman2023concentration, Theorem 2.2
  • Corollary 7
  • Theorem 8: Additive form
  • Theorem 9: Multiplicative form
  • Corollary 10: Multiplicative form; spectral
  • Remark 11: Poincaré constant of $\nu$
  • ...and 47 more