Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Upper and lower bounds on TVD and KLD between centered elliptical distributions in high-dimensional setting

Ievlev Pavel, Timofei Shashkov

TL;DR

This work develops high‑dimensional bounds on the total variation distance $D_{\,mathrm{TV}}$ and the Kullback–Leibler divergence $D_{\,mathrm{KL}}$ between centered elliptical distributions. A covariance‑reduction approach reduces general comparisons to the identity and diagonal covariance cases, enabling bounds expressed through incomplete Gamma/Beta functions and the Lambert $W$‑function. The authors instantiate these bounds for the multivariate Student‑t versus multivariate normal case, and for independent‑component Gamma laws (the latter focusing on TVD), providing explicit formulas and computable thresholds that hold in large $n$. The results facilitate robust, tractable comparisons in high dimensions when exact densities are unavailable, with potential implications for hypothesis testing, privacy, and model selection. The methodology hinges on $f$‑divergence techniques, sublevel set analyses, and CLT‑style approximations, yielding practically usable bounds even as the dimension grows.

Abstract

In this paper, we derive some upper and lower bounds and inequalities for the total variation distance (TVD) and the Kullback-Leibler divergence (KLD), also known as the relative entropy, between two probability measures $μ$ and $ν$ defined by $$ D_{\mathrm{TV}} ( μ, ν) = \sup_{B \in \mathcal{B} (\mathbb{R}^n)} \left| μ(B) - ν(B) \right| \quad \text{and} \quad D_{\mathrm{KL}} ( μ\, \| \, ν) = \int_{\mathbb{R}^n} \ln \left( \frac{dμ(x)}{dν(x)} \right) \, μ(dx) $$ correspondingly when the dimension $n$ is high. We begin with some elementary bounds for centered elliptical distributions admitting densities and showcase how these bounds may be used by estimating the TVD and KLD between multivariate Student and multivariate normal distribution in the high-dimensional setting. Next, we show how the same approach simplifies when we apply it to multivariate Gamma distributions with independent components (in the latter case, we only study the TVD, because KLD may be calculated explicitly, see [1]). Our approach is motivated by the recent contribution by Barabesi and Pratelli [2].

Upper and lower bounds on TVD and KLD between centered elliptical distributions in high-dimensional setting

TL;DR

This work develops high‑dimensional bounds on the total variation distance and the Kullback–Leibler divergence between centered elliptical distributions. A covariance‑reduction approach reduces general comparisons to the identity and diagonal covariance cases, enabling bounds expressed through incomplete Gamma/Beta functions and the Lambert ‑function. The authors instantiate these bounds for the multivariate Student‑t versus multivariate normal case, and for independent‑component Gamma laws (the latter focusing on TVD), providing explicit formulas and computable thresholds that hold in large . The results facilitate robust, tractable comparisons in high dimensions when exact densities are unavailable, with potential implications for hypothesis testing, privacy, and model selection. The methodology hinges on ‑divergence techniques, sublevel set analyses, and CLT‑style approximations, yielding practically usable bounds even as the dimension grows.

Abstract

In this paper, we derive some upper and lower bounds and inequalities for the total variation distance (TVD) and the Kullback-Leibler divergence (KLD), also known as the relative entropy, between two probability measures and defined by correspondingly when the dimension is high. We begin with some elementary bounds for centered elliptical distributions admitting densities and showcase how these bounds may be used by estimating the TVD and KLD between multivariate Student and multivariate normal distribution in the high-dimensional setting. Next, we show how the same approach simplifies when we apply it to multivariate Gamma distributions with independent components (in the latter case, we only study the TVD, because KLD may be calculated explicitly, see [1]). Our approach is motivated by the recent contribution by Barabesi and Pratelli [2].
Paper Structure (10 sections, 13 theorems, 40 equations, 1 figure)

This paper contains 10 sections, 13 theorems, 40 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Elementary bounds on TVD and KLD between two elliptical distributions
  3. Multivariate Student t-distribution against multivariate normal distribution
  4. Total variation distance
  5. KL divergence
  6. Distances between two Gamma distributions
  7. Appendix
  8. Proof of Lemma \ref{['covariance_reduction']}.
  9. Proof of Proposition \ref{['tv_gen']}
  10. Proof of Proposition \ref{['kl_gen']}

Key Result

Lemma 1

Let $D$ be the eigenvalue matrix of $\Sigma_2^{-1} \Sigma_1$. Then where $D_{f}$ is the $f$-divergence associated to a convex function $f \colon [0, \infty) \to (-\infty, \infty]$ such that $f(x) < \infty$ for all $x > 0$ and $f(1) = 0$ by

Figures (1)

  • Figure 1: These plots illustrate the lower and upper bounds behaviour for three different vectors $d = (d_i)_{i = 1, \dots , 800}$, randomly chosen from $[0.95, 1.01]^{800}$.

Theorems & Definitions (21)

  • Lemma 1
  • Proposition 1
  • Proposition 2
  • Lemma 2
  • Remark 1
  • Lemma 3
  • Lemma 4
  • proof : Proof of \ref{['n0']}
  • Remark 2
  • Lemma 5
  • ...and 11 more