Derivatives of entropy and the MMSE conjecture

TL;DR

This work analyzes derivatives of the entropy along the heat flow, linking $H(X,t)$ and its derivatives to the Fisher information via the De Bruijn identity and to cumulants through Ledoux's $\Gamma$-calculus framework. It develops a fine combinatorial analysis of the inductive structure of the polynomials $R_n$ that appear in $\partial_t^n H(X,t)$ and obtains closed formulas for their leading coefficients, supported by an algorithm (AlgoG) for moderate orders. Leveraging these explicit leading-term formulas, the authors revisit the MMSE conjecture, establishing new cumulant-driven conditions under which equality of the mmse functions implies equality of laws (up to translation or reflection); they also provide corollaries for uniform and Rademacher distributions and discuss positivity/sign conditions on cumulants. The results deepen the connection between cumulant dynamics, entropy derivatives, and identifiability in Gaussian-noise models, offering concrete criteria to ensure unique law reconstruction from information-theoretic derivatives along the heat flow.

Abstract

We investigate the entropy $H(μ,t)$ of a probability measure $μ$ along the heat flow and more precisely we seek for closed algebraic representations of its derivatives. Provided that $μ$ admits moments of any order, it is indeed proved in [Guo et al., 2010] that $t\mapsto H(μ,t)$ is smooth, and in [Ledoux, 2016] that its derivatives at zero can be expressed into multivariate polynomials evaluated in the moments (or cumulants) of $μ$. In the seminal contribution \cite{Led}, these algebraic expressions are derived through $Γ$-calculus techniques which provide implicit recursive formulas for these polynomials. Our main contribution consists in a fine combinatorial analysis of these inductive relations and for the first time to derive closed formulas for the leading coefficients of these polynomials expressions. Building upon these explicit formulas we revisit the so-called "MMSE conjecture" from [Guo et al., 2010] which asserts that two distributions on the real line with the same entropy along the heat flow must coincide up to translation and symmetry. Our approach enables us to provide new conditions on the source distributions ensuring that the MMSE conjecture holds and to refine several criteria proved in [Ledoux, 2016]. As illustrating examples, our findings cover the cases of uniform and Rademacher distributions, for which previous results in the literature were inapplicable.

Theorems & Definitions (81)

• Theorem 1.1: Theorem 1, Led
• Theorem 1.2
• Remark 1.3
• Remark 1.4
• Remark 1.5
• Example 1.6
• Definition 2.1
• Remark 2.2: When is a law moment-determined?
• Definition 2.3
• Proposition 2.4