On the monotonicity of discrete entropy for log-concave random vectors on $\mathbb{Z}^d$

TL;DR

The paper establishes high-dimensional discrete entropy monotonicity for sums of i.i.d. isotropic, log-concave vectors on $\mathbb{Z}^d$, proving $H(X_1+\cdots+X_{n+1}) \ge H(X_1+\cdots+X_n) + \frac{d}{2}\log\Bigl(\frac{n+1}{n}\Bigr) - o(1)$ with an explicit rate $o(1)=O\big(H(X_1) e^{-{H(X_1)}/d}\big)$. The approach couples discrete entropy to differential entropy by adding i.i.d. uniform dithers $U_i$ on $[0,1]^d$ and applying the continuous EPI, then shows the discrete-distribution tails and smoothness conditions are preserved under convolution. A substantial portion is devoted to verifying these regularity conditions for discrete log-concave distributions in $\mathbb{Z}^d$ using convex geometric tools (Ball's bodies, isotropic constants) and a relaxation to almost isotropicity. As a result, the results extend the 1D discrete monotonicity to higher dimensions and broaden the admissible distribution classes while quantifying the discretization error in terms of the discrete entropy $H(X_1)$. The paper also provides a discrete-to-continuous entropy approximation with the same rate and discusses open questions about the necessity of isotropic extensions.

Abstract

We prove the following type of discrete entropy monotonicity for sums of isotropic, log-concave, independent and identically distributed random vectors $X_1,\dots,X_{n+1}$ on $\mathbb{Z}^d$: $$ H(X_1+\cdots+X_{n+1}) \geq H(X_1+\cdots+X_{n}) + \frac{d}{2}\log{\Bigl(\frac{n+1}{n}\Bigr)} +o(1), $$ where $o(1)$ vanishes as $H(X_1) \to \infty$. Moreover, for the $o(1)$-term, we obtain a rate of convergence $ O\Bigl({H(X_1)}{e^{-\frac{1}{d}H(X_1)}}\Bigr)$, where the implied constants depend on $d$ and $n$. This generalizes to $\mathbb{Z}^d$ the one-dimensional result of the second named author (2023). As in dimension one, our strategy is to establish that the discrete entropy $H(X_1+\cdots+X_{n})$ is close to the differential (continuous) entropy $h(X_1+U_1+\cdots+X_{n}+U_{n})$, where $U_1,\dots, U_n$ are independent and identically distributed uniform random vectors on $[0,1]^d$ and to apply the theorem of Artstein, Ball, Barthe and Naor (2004) on the monotonicity of differential entropy. In fact, we show this result under more general assumptions than log-concavity, which are preserved up to constants under convolution. In order to show that log-concave distributions satisfy our assumptions in dimension $d\ge2$, more involved tools from convex geometry are needed because a suitable position is required. We show that, for a log-concave function on $\mathbb{R}^d$ in isotropic position, its integral, barycenter and covariance matrix are close to their discrete counterparts. Moreover, in the log-concave case, we weaken the isotropicity assumption to what we call almost isotropicity. One of our technical tools is a discrete analogue to the upper bound on the isotropic constant of a log-concave function, which extends to dimensions $d\ge1$ a result of Bobkov, Marsiglietti and Melbourne (2022).

Theorems & Definitions (60)

• Theorem 1: gavalakis
• Definition 2
• Theorem 3
• Corollary 4
• proof
• Theorem 5
• Remark 6
• Theorem 7
• Theorem 8
• Theorem 9