Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On sample complexity for covariance estimation via the unadjusted Langevin algorithm

Shogo Nakakita

TL;DR

This work analyzes covariance estimation for strongly log-concave, smooth distributions using the unadjusted Langevin algorithm (ULA). It introduces a concentration bound for the dependent sample covariance via a log-Sobolev inequality on the joint ULA iterates, yielding a variance rate of order $O\left(\frac{1}{\alpha}\sqrt{\frac{d}{\alpha \eta n}}\right)$ and an explicit bias–variance decomposition. By combining this with existing discretization bias bounds, the authors derive non-asymptotic sample complexities for estimating Cov(π) with ULA, and compare against embarrassingly parallel ULA, showing a logarithmic advantage for the single-chain approach due to burn-in bias reduction. A key technical contribution is establishing a joint log-Sobolev inequality with constant $2/(\alpha^2 \eta)$ for the sequence $(X_{m+1},\ldots,X_{m+n})$, which underpins the concentration results. The findings quantify how burn-in and dependence affect sample complexity and provide practical guidance on the trade-offs between single-chain and parallel ULA in high-dimensional covariance estimation.

Abstract

We establish sample complexity guarantees for estimating the covariance matrix of strongly log-concave smooth distributions using the unadjusted Langevin algorithm (ULA). We quantitatively compare our complexity estimates on single-chain ULA with embarrassingly parallel ULA and derive that the sample complexity of the single-chain approach is smaller than that of embarrassingly parallel ULA by a logarithmic factor in the dimension and the reciprocal of the prescribed precision, with the difference arising from effective bias reduction through burn-in. The key technical contribution is a concentration bound for the sample covariance matrix around its expectation, derived via a log-Sobolev inequality for the joint distribution of ULA iterates.

On sample complexity for covariance estimation via the unadjusted Langevin algorithm

TL;DR

This work analyzes covariance estimation for strongly log-concave, smooth distributions using the unadjusted Langevin algorithm (ULA). It introduces a concentration bound for the dependent sample covariance via a log-Sobolev inequality on the joint ULA iterates, yielding a variance rate of order and an explicit bias–variance decomposition. By combining this with existing discretization bias bounds, the authors derive non-asymptotic sample complexities for estimating Cov(π) with ULA, and compare against embarrassingly parallel ULA, showing a logarithmic advantage for the single-chain approach due to burn-in bias reduction. A key technical contribution is establishing a joint log-Sobolev inequality with constant for the sequence , which underpins the concentration results. The findings quantify how burn-in and dependence affect sample complexity and provide practical guidance on the trade-offs between single-chain and parallel ULA in high-dimensional covariance estimation.

Abstract

We establish sample complexity guarantees for estimating the covariance matrix of strongly log-concave smooth distributions using the unadjusted Langevin algorithm (ULA). We quantitatively compare our complexity estimates on single-chain ULA with embarrassingly parallel ULA and derive that the sample complexity of the single-chain approach is smaller than that of embarrassingly parallel ULA by a logarithmic factor in the dimension and the reciprocal of the prescribed precision, with the difference arising from effective bias reduction through burn-in. The key technical contribution is a concentration bound for the sample covariance matrix around its expectation, derived via a log-Sobolev inequality for the joint distribution of ULA iterates.
Paper Structure (18 sections, 15 theorems, 73 equations)

This paper contains 18 sections, 15 theorems, 73 equations.

Table of Contents

  1. Introduction
  2. Problem setting
  3. Main contributions
  4. Literature review
  5. Notation
  6. Paper organization
  7. Concentration on the sample covariance matrix of the unadjusted Langevin algorithm
  8. Setup
  9. Concentration bound
  10. Technical remarks on Theorem \ref{['thm:concentration']}
  11. Sample complexity guarantees for covariance estimation
  12. Sample complexity for the unadjusted Langevin algorithm
  13. Comparison with embarrassingly parallel ULA
  14. Proofs
  15. Preliminaries
  16. ...and 3 more sections

Key Result

Theorem 1

Suppose that Assumption asmp:potential holds. Assume that $\eta\in(0,1/\beta)$ and the distribution of $X_{0}$ satisfies a log-Sobolev inequality with constant $\kappa_{0}$. If (with convention $\log0=-\infty$), then for any $\delta>0$, with probability at least $1-4\delta$, In particular, $\|\hat{\Sigma}_{n}-\mathbb{E}[\hat{\Sigma}_{n}]\|=\mathcal{O}((1/\alpha)(\sqrt{(d+\log\delta^{-1})/(\alpha

Theorems & Definitions (29)

  • Definition 1
  • Theorem 1
  • Proposition 2
  • Proposition 3
  • Proposition 4: LSI for joint distributions
  • Remark 1
  • proof
  • Lemma 5: Subexponentiality of quadratic forms; nakakita2025corrigendum
  • Lemma 6: LSI for marginal distributions
  • proof
  • ...and 19 more