Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The strong data processing inequality under the heat flow

Bo'az Klartag, Or Ordentlich

TL;DR

The paper analyzes how divergences between two distributions evolve under the heat flow induced by Gaussian noise, formalizing a strong data-processing inequality with contraction coefficients η_φ(μ,s). By connecting η_χ^2 and η_KL to functional-inequality constants C_P(μ) and C_LS(μ), respectively, the authors derive explicit bounds and convexity properties, generalize de Bruijn identities to φ-divergences, and establish MMSE- and I-MMSE-based bounds that link estimation error to information loss. They introduce φ-Sobolev constants to unify contraction bounds across a broad family of divergences and prove subadditivity and Gaussian-extremality results that anchor the Gaussian case as a benchmark. The results yield practical insights for the Gaussian channel Y = X + √s Z, including half-blurring times and information-theoretic bounds that tighten previous EPI-based estimates, with implications for estimation, concentration, and information geometry. Overall, the work advances a cohesive framework tying heat-flow evolution, SDPI, and functional inequalities to quantitative bounds on information measures under Gaussian perturbations.

Abstract

Let $ν$ and $μ$ be probability distributions on $\mathbb{R}^n$, and $ν_s,μ_s$ be their evolution under the heat flow, that is, the probability distributions resulting from convolving their density with the density of an isotropic Gaussian random vector with variance $s$ in each entry. This paper studies the rate of decay of $s\mapsto D(ν_s\|μ_s)$ for various divergences, including the $χ^2$ and Kullback-Leibler (KL) divergences. We prove upper and lower bounds on the strong data-processing inequality (SDPI) coefficients corresponding to the source $μ$ and the Gaussian channel. We also prove generalizations of de Brujin's identity, and Costa's result on the concavity in $s$ of the differential entropy of $ν_s$. As a byproduct of our analysis, we obtain new lower bounds on the mutual information between $X$ and $Y=X+\sqrt{s} Z$, where $Z$ is a standard Gaussian vector in $\mathbb{R}^n$, independent of $X$, and on the minimum mean-square error (MMSE) in estimating $X$ from $Y$, in terms of the Poincaré constant of $X$.

The strong data processing inequality under the heat flow

TL;DR

The paper analyzes how divergences between two distributions evolve under the heat flow induced by Gaussian noise, formalizing a strong data-processing inequality with contraction coefficients η_φ(μ,s). By connecting η_χ^2 and η_KL to functional-inequality constants C_P(μ) and C_LS(μ), respectively, the authors derive explicit bounds and convexity properties, generalize de Bruijn identities to φ-divergences, and establish MMSE- and I-MMSE-based bounds that link estimation error to information loss. They introduce φ-Sobolev constants to unify contraction bounds across a broad family of divergences and prove subadditivity and Gaussian-extremality results that anchor the Gaussian case as a benchmark. The results yield practical insights for the Gaussian channel Y = X + √s Z, including half-blurring times and information-theoretic bounds that tighten previous EPI-based estimates, with implications for estimation, concentration, and information geometry. Overall, the work advances a cohesive framework tying heat-flow evolution, SDPI, and functional inequalities to quantitative bounds on information measures under Gaussian perturbations.

Abstract

Let and be probability distributions on , and be their evolution under the heat flow, that is, the probability distributions resulting from convolving their density with the density of an isotropic Gaussian random vector with variance in each entry. This paper studies the rate of decay of for various divergences, including the and Kullback-Leibler (KL) divergences. We prove upper and lower bounds on the strong data-processing inequality (SDPI) coefficients corresponding to the source and the Gaussian channel. We also prove generalizations of de Brujin's identity, and Costa's result on the concavity in of the differential entropy of . As a byproduct of our analysis, we obtain new lower bounds on the mutual information between and , where is a standard Gaussian vector in , independent of , and on the minimum mean-square error (MMSE) in estimating from , in terms of the Poincaré constant of .
Paper Structure (16 sections, 19 theorems, 176 equations)

This paper contains 16 sections, 19 theorems, 176 equations.

Table of Contents

  1. Introduction
  2. Main results on $\chi^2$ divergence
  3. Main results on KL-divergence
  4. Main results on general divergences
  5. Related Work
  6. Minimum-squared error and mutual information
  7. Proof of the lower bound in Theorem \ref{['thm:etachi2UB']}
  8. Proof of Corollary \ref{['cor:mi']}
  9. $\chi^2$-divergence and proofs for blurring time bounds
  10. Identities for KL-divergence and general divergences
  11. Proofs of inequalities related to $\varphi$-Sobolev constants
  12. Technical justifications for Lemma \ref{['lem_620']}
  13. Justification of first integration in parts
  14. Justification of second integration in parts
  15. Justification of taking derivative under the integral sign
  16. ...and 1 more sections

Key Result

Theorem 1.1

For any probability distribution $\mu$ and $s>0$ If $\mu$ is additionally log-concave, then,

Theorems & Definitions (34)

  • Theorem 1.1
  • Theorem 1.2
  • Corollary 1.3
  • Corollary 1.4
  • Proposition 1.5
  • Theorem 1.6
  • Theorem 1.7
  • Proposition 1.8
  • Proposition 1.9
  • Proposition 1.10
  • ...and 24 more