A Quantitative Entropy Power Inequality for Dependent Random Vectors
Mokshay Madiman, James Melbourne, Cyril Roberto
TL;DR
This paper extends the entropy power inequality (EPI) to dependent random vectors by introducing a framework based on log-supermodular joint densities and an Ornstein–Uhlenbeck flow. It develops a generalized Fisher information inequality for an arbitrary number of summands, and derives a linearized EPI with a quantitative correction term that captures dependence. It then establishes an EPI for conditional entropy and identifies log-supermodularity (and its Gaussian-convolution stability) as a sufficient condition yielding a clean, unconditional conditional-EPI bound. The results connect to and extend prior work by Rioul and Hao–Jog, offering a unified, quantitative view of EPI under dependence with potential implications for probability, information theory, and related inequalities.
Abstract
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been developed for settings with dependence, including by Takano, Johnson, and Rioul. We extend these works by developing a quantitative version of the entropy power inequality for dependent random vectors. A notable consequence is that an entropy power inequality stated using conditional entropies holds for random vectors whose joint density is log-supermodular.