Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sharp Inequalities between Total Variation and Hellinger Distances for Gaussian Mixtures

Joonhyuk Jung, Chao Gao

TL;DR

This work resolves a fundamental question about the relationship between total variation and Hellinger distances for Gaussian location mixtures. It introduces a general bound H(f_π,f_η) ≤ (C_0 ∨ TV^{−α(TV)}) TV, with α(t) = (2+δ)/log(log(1/t) ∨ e), and proves near-tightness via carefully constructed sequence pairs, using a Hermite expansion framework alongside Nikolskii-type and restricted-range inequalities. The results yield an entropic characterization of learning Gaussian mixtures in TV, paralleling Hellinger-KL relations, and provide sharp robust density estimation guarantees under HubER contamination via the Yatracos estimator and regularized empirical Bayes Tweedie estimators. Practically, these findings inform minimax rates and robust estimation strategies for Gaussian mixtures in settings with outliers, and they resolve an open problem in the literature. The work also outlines several natural extensions, including heteroscedastic mixtures and connections to L^2 and Fisher divergences, which have implications for nonparametric density estimation and robust inference.

Abstract

We study the relation between the total variation (TV) and Hellinger distances between two Gaussian location mixtures. Our first result establishes a general upper bound: for any two mixing distributions supported on a compact set, the Hellinger distance between the two mixtures is controlled by the TV distance raised to a power $1-o(1)$, where the $o(1)$ term is of order $1/\log\log(1/\mathrm{TV})$. We also construct two sequences of mixing distributions that demonstrate the sharpness of this bound. Taken together, our results resolve an open problem raised in Jia et al. (2023) and thus lead to an entropic characterization of learning Gaussian mixtures in total variation. Our inequality also yields optimal robust estimation of Gaussian mixtures in Hellinger distance, which has a direct implication for bounding the minimax regret of empirical Bayes under Huber contamination.

Sharp Inequalities between Total Variation and Hellinger Distances for Gaussian Mixtures

TL;DR

This work resolves a fundamental question about the relationship between total variation and Hellinger distances for Gaussian location mixtures. It introduces a general bound H(f_π,f_η) ≤ (C_0 ∨ TV^{−α(TV)}) TV, with α(t) = (2+δ)/log(log(1/t) ∨ e), and proves near-tightness via carefully constructed sequence pairs, using a Hermite expansion framework alongside Nikolskii-type and restricted-range inequalities. The results yield an entropic characterization of learning Gaussian mixtures in TV, paralleling Hellinger-KL relations, and provide sharp robust density estimation guarantees under HubER contamination via the Yatracos estimator and regularized empirical Bayes Tweedie estimators. Practically, these findings inform minimax rates and robust estimation strategies for Gaussian mixtures in settings with outliers, and they resolve an open problem in the literature. The work also outlines several natural extensions, including heteroscedastic mixtures and connections to L^2 and Fisher divergences, which have implications for nonparametric density estimation and robust inference.

Abstract

We study the relation between the total variation (TV) and Hellinger distances between two Gaussian location mixtures. Our first result establishes a general upper bound: for any two mixing distributions supported on a compact set, the Hellinger distance between the two mixtures is controlled by the TV distance raised to a power , where the term is of order . We also construct two sequences of mixing distributions that demonstrate the sharpness of this bound. Taken together, our results resolve an open problem raised in Jia et al. (2023) and thus lead to an entropic characterization of learning Gaussian mixtures in total variation. Our inequality also yields optimal robust estimation of Gaussian mixtures in Hellinger distance, which has a direct implication for bounding the minimax regret of empirical Bayes under Huber contamination.
Paper Structure (20 sections, 30 theorems, 169 equations)

This paper contains 20 sections, 30 theorems, 169 equations.

Table of Contents

  1. Introduction
  2. Paper Organization
  3. Notation
  4. Main Results
  5. Sharpness
  6. Applications
  7. Entropic Characterization of Learning in TV
  8. Robust Density Estimation
  9. Discussion
  10. Proof of the Main Results
  11. Preliminaries: Hermite Polynomials and Inequalities
  12. Proof of the Main Theorem
  13. Dependency of the Constant
  14. Physical Interpretation: Quantum Harmonic Oscillator
  15. Proof of the Sharpness
  16. ...and 5 more sections

Key Result

Theorem 2.1

Let $\pi$ and $\eta$ be probability measures supported on the $d$-dimensional cube $[-M, M]^d$. Let $\delta > 0$. Then, there exists $C_0 = C_0(\delta, M, d) > 0$, not depending on $\pi$ or $\eta$, such that where we define for $t > 0$.

Theorems & Definitions (66)

  • Theorem 2.1: Inequality between TV distance and $\chi^2$-divergence
  • Remark 2.2
  • Remark 2.3
  • Corollary 2.4: Inequality between TV and Hellinger distances
  • proof
  • Theorem 2.5: Inequality between $L^1(\phi_d)$ and $L^2(\phi_d)$ norms
  • proof
  • proof : Proof of Theorem \ref{['theorem:uniformTV']}
  • Theorem 3.1: Sharpness of Corollary \ref{['corollary:uniformTV']}
  • Lemma 3.2: Sharp example of Theorem \ref{['theorem:uniformnorm']}
  • ...and 56 more