Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The total variation distance between high-dimensional Gaussians with the same mean

Luc Devroye, Abbas Mehrabian, Tommy Reddad

TL;DR

The paper derives tight, constant-factor bounds for the total variation distance between high-dimensional Gaussians with the same mean, expressed via the eigenvalues of $\Sigma_1^{-1}\Sigma_2 - I_d$ and, in the reduced-rank case, via eigenvalues of projected covariance comparisons. It leverages coupling, KL-divergence, and Hellinger-distance techniques to obtain computable upper and lower bounds, and provides dedicated analyses for the one-dimensional and general cases. The results yield concrete guidance on how covariance structure influences discriminability of Gaussian models in high dimensions, with extensions to singular covariances through subspace projections. An open problem remains to characterize TV bounds for Gaussians with different means in closed form.

Abstract

Given two high-dimensional Gaussians with the same mean, we prove a lower and an upper bound for their total variation distance, which are within a constant factor of one another.

The total variation distance between high-dimensional Gaussians with the same mean

TL;DR

The paper derives tight, constant-factor bounds for the total variation distance between high-dimensional Gaussians with the same mean, expressed via the eigenvalues of and, in the reduced-rank case, via eigenvalues of projected covariance comparisons. It leverages coupling, KL-divergence, and Hellinger-distance techniques to obtain computable upper and lower bounds, and provides dedicated analyses for the one-dimensional and general cases. The results yield concrete guidance on how covariance structure influences discriminability of Gaussian models in high dimensions, with extensions to singular covariances through subspace projections. An open problem remains to characterize TV bounds for Gaussians with different means in closed form.

Abstract

Given two high-dimensional Gaussians with the same mean, we prove a lower and an upper bound for their total variation distance, which are within a constant factor of one another.

Paper Structure

This paper contains 10 sections, 7 theorems, 57 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Matrix definitions.
  4. The coupling characterization of the total variation distance.
  5. An upper bound for the total variation distance using the KL-divergence.
  6. Bounds for the total variation distance using the Hellinger distance.
  7. The same-mean case: proof of Theorem \ref{['thm:meanzero']}
  8. The one-dimensional case: proof of Theorem \ref{['thm:onedimensional']}
  9. The general case: proof of Theorem \ref{['thm:main']}
  10. Acknowledgments.

Key Result

Theorem 1.1

Let $\mu\in \mathbb{R}^d$, $\Sigma_1$ and $\Sigma_2$ be positive definite $d\times d$ matrices, and $\lambda_1,\dots,\lambda_d$ denote the eigenvalues of $\Sigma_1^{-1}\Sigma_2-I_d$. Then, If $\Sigma_1$ and $\Sigma_2$ are positive semi-definite, $\operatorname{range}(\Sigma_1)=\operatorname{range}(\Sigma_2)$, and $r = \operatorname{rank}(\Sigma_1) = \operatorname{rank}(\Sigma_2)$, then let $\Pi$

Theorems & Definitions (10)

  • Theorem 1.1: Total variation distance between Gaussians with the same mean
  • Theorem 1.2: Total variation distance between Gaussians with different means
  • Theorem 1.3: Total variation distance between one-dimensional Gaussians
  • Proposition 2.1
  • Proposition 2.2
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • proof : Proof of Theorem \ref{['thm:meanzero']}