Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimum entropy of a log-concave variable for fixed variance

James Melbourne, Piotr Nayar, Cyril Roberto

TL;DR

The paper proves that among log-concave real random variables with fixed variance, the Shannon differential entropy is minimized by the standard one-sided exponential distribution, with the tight bound $h(X) \ge \frac{1}{2}\log\mathrm{Var}(X) + 1$ and equality at $X\sim\mathrm{Exp}(1)$. The authors develop a systematic reduction based on decreasing rearrangement, degrees of freedom, and localization to confine the extremal problem to monotone two-piece log-affine densities, then verify the inequality through a three-point inequality and a careful polynomial decomposition that relies on the positivity of coefficient polynomials $P_i$. Consequences include optimal bounds for additive-noise channel capacities under log-concave noise, improved reverse entropy power inequalities, and extended Rényi-entropy-type bounds, providing tight, broadly applicable constants. The results yield a principled, geometry-inspired approach to entropy minimization in constrained log-concave settings, with implications for both information theory and probabilistic inequalities.

Abstract

We show that for log-concave real random variables with fixed variance the Shannon differential entropy is minimized for an exponential random variable. We apply this result to derive upper bounds on capacities of additive noise channels with log-concave noise. We also improve constants in the reverse entropy power inequalities for log-concave random variables.

Minimum entropy of a log-concave variable for fixed variance

TL;DR

The paper proves that among log-concave real random variables with fixed variance, the Shannon differential entropy is minimized by the standard one-sided exponential distribution, with the tight bound and equality at . The authors develop a systematic reduction based on decreasing rearrangement, degrees of freedom, and localization to confine the extremal problem to monotone two-piece log-affine densities, then verify the inequality through a three-point inequality and a careful polynomial decomposition that relies on the positivity of coefficient polynomials . Consequences include optimal bounds for additive-noise channel capacities under log-concave noise, improved reverse entropy power inequalities, and extended Rényi-entropy-type bounds, providing tight, broadly applicable constants. The results yield a principled, geometry-inspired approach to entropy minimization in constrained log-concave settings, with implications for both information theory and probabilistic inequalities.

Abstract

We show that for log-concave real random variables with fixed variance the Shannon differential entropy is minimized for an exponential random variable. We apply this result to derive upper bounds on capacities of additive noise channels with log-concave noise. We also improve constants in the reverse entropy power inequalities for log-concave random variables.
Paper Structure (17 sections, 15 theorems, 90 equations)

This paper contains 17 sections, 15 theorems, 90 equations.

Table of Contents

  1. Introduction
  2. Applications
  3. Additive noise channels
  4. Reverse EPI
  5. Reductions
  6. Decreasing Rearrangement
  7. Degrees of freedom
  8. Localization
  9. Proof of Theorem \ref{['thm:main']}
  10. Three-point inequality
  11. Estimating the exponent
  12. Positivity of $P_i(x,y)$, $i=0,1,\dots, 4$
  13. Positivity of $P_4(x,y)$
  14. Positivity of $P_0(x,y)$
  15. Positivity of $P_1(x,y)$
  16. ...and 2 more sections

Key Result

Theorem 1.1

For a log-concave random variable $X$ we have with equality for the standard one-sided exponential random variable with density $e^{-x}\mathbf{1}_{[0,\infty)}(x)$.

Theorems & Definitions (25)

  • Theorem 1.1
  • Corollary 1.2
  • Proposition 2.1
  • Corollary 2.2
  • Corollary 2.3
  • Corollary 2.4
  • Definition 3.1: Decreasing Rearrangement
  • Definition 3.2
  • Proposition 3.3
  • proof
  • ...and 15 more