Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of Differential Entropies -- II

Mahesh Godavarti

Abstract

We show that under convergence in measure of probability density functions, differential entropy converges whenever the entropy integrands $f_n |\log f_n|$ are uniformly integrable and tight -- a direct consequence of Vitali's convergence theorem. We give an entropy-weighted Orlicz condition: $\sup_n \int f_n\, Ψ(|\log f_n|) < \infty$ for a single superlinear~$Ψ$, strictly weaker than the fixed-$α$ condition of Godavarti and Hero (2004). We also disprove the Godavarti-Hero conjecture that $α> 1$ could be replaced by $α_n \downarrow 1$. We recover the sufficient conditions of Godavarti--Hero, Piera--Parada, and Ghourchian-Gohari-Amini as corollaries. On bounded domains, we prove that uniform integrability of the entropy integrands is both necessary and sufficient -- a complete characterization of entropy convergence.

Convergence of Differential Entropies -- II

Abstract

We show that under convergence in measure of probability density functions, differential entropy converges whenever the entropy integrands are uniformly integrable and tight -- a direct consequence of Vitali's convergence theorem. We give an entropy-weighted Orlicz condition: for a single superlinear~, strictly weaker than the fixed- condition of Godavarti and Hero (2004). We also disprove the Godavarti-Hero conjecture that could be replaced by . We recover the sufficient conditions of Godavarti--Hero, Piera--Parada, and Ghourchian-Gohari-Amini as corollaries. On bounded domains, we prove that uniform integrability of the entropy integrands is both necessary and sufficient -- a complete characterization of entropy convergence.
Paper Structure (14 sections, 11 theorems, 9 equations, 2 tables)

This paper contains 14 sections, 11 theorems, 9 equations, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Uniform integrability and tightness
  5. Classical tools
  6. Densities under convergence in measure
  7. Entropy convergence via Vitali's theorem
  8. The Orlicz Condition
  9. Unification of Existing Results
  10. Recovering Godavarti--Hero
  11. Recovering Piera--Parada
  12. Recovering Ghourchian--Gohari--Amini
  13. The GH Conjecture is False
  14. Conclusion

Key Result

Theorem 2

A family $\{g_n\}$ on a finite measure space is uniformly integrable if and only if there exists a convex $\Phi : [0,\infty) \to [0,\infty)$ with $\Phi(0) = 0$ and $\Phi(t)/t \to \infty$ such that $\sup_n \int \Phi(\lvert g_n \rvert) \, d\mu < \infty$.

Theorems & Definitions (24)

  • Definition 1: UI & T
  • Theorem 2: de la Vallée--Poussin delaValleePoussin1915
  • Theorem 3: Vitali; see Folland1999
  • Lemma 4
  • proof
  • Lemma 5: Entropy convergence via UI & T
  • proof
  • Example 6: Converse fails on $\mathbb{R}^d$
  • Theorem 7: Bounded domains
  • proof
  • ...and 14 more