Properties of the Strong Data Processing Constant for Rényi Divergence

TL;DR

This work analyzes the Strong Data Processing Inequality (SDPI) constants for Rényi divergences, highlighting that unlike $\Phi$-divergences, Rényi SDPI constants do not admit a universal upper bound in terms of total variation but do satisfy a universal lower bound $\eta_{\alpha}(K) \geq \eta_{\chi^2}(K)$. It introduces two complementary lower bounds for $\eta_{\alpha}(\mu,K)$—a universal second-order bound and a computable finite-alphabet bound—and shows scenarios where they coincide. The paper also specializes to $\alpha=2$, proving that the supremum over $\nu$ is achieved on boundary distributions (with at least one zero probability), and provides a closed-form characterization for binary channels, including explicit expressions for the binary symmetric channel. These results have implications for contraction properties, concentration, and Bayesian risk in privatized or dependent settings, and connect to related functional-analytic tools such as log-Sobolev inequalities and hypercontractivity.

Abstract

Strong data processing inequalities (SDPI) are an important object of study in Information Theory and have been well studied for $f$-divergences. Universal upper and lower bounds have been provided along with several applications, connecting them to impossibility (converse) results, concentration of measure, hypercontractivity, and so on. In this paper, we study Rényi divergence and the corresponding SDPI constant whose behavior seems to deviate from that of ordinary $Φ$-divergences. In particular, one can find examples showing that the universal upper bound relating its SDPI constant to the one of Total Variation does not hold in general. In this work, we prove, however, that the universal lower bound involving the SDPI constant of the Chi-square divergence does indeed hold. Furthermore, we also provide a characterization of the distribution that achieves the supremum when $α$ is equal to $2$ and consequently compute the SDPI constant for Rényi divergence of the general binary channel.

Figures (1)

• Figure 1: Plots for $\eta_{\chi^2}(\text{BSC}(\varepsilon))$, $\eta_{2}(\text{BSC}(\varepsilon))$ and $\eta_{\text{TV}}(\text{BSC}(\varepsilon))$.

Theorems & Definitions (14)

• Theorem 1: Lower bound, Version 1
• proof
• Corollary 2
• proof
• Theorem 3: Lower bound, Version 2
• proof
• Example 1
• Example 2
• Theorem 4
• proof