Contraction of Rényi Divergences for Discrete Channels: Properties and Applications

TL;DR

This paper analyzes contraction properties of Rényi divergences under Markov kernels via Strong Data-Processing Inequalities (SDPIs), revealing a critical dependence on the order $\alpha$. By linking $D_{\alpha}$ to the Hellinger family through $D_{\alpha}(\nu\|\mu)=\frac{1}{\alpha-1}\log\bigl(1+(\alpha-1)\mathcal{H}_{\alpha}(\nu\|\mu)\bigr)$, it derives $\alpha$-dependent inequalities that compare $\eta_{\alpha}$ with $\eta_{\mathcal{H}_{\alpha}}$ and $\eta_{\chi^2}$, and identifies a dichotomy: for $\alpha\in[0,1]$ the behavior mirrors that of $\varphi$-divergences, while for $\alpha>1$ striking deviations occur. A key contribution is the explicit characterization of $\eta_{\infty}$ and the distribution-independent form $\eta_{\infty}(K)=\sup_{x,x'}\bigl[1-\min_y \frac{K(y|x')}{K(y|x)}\bigr]$, with connections to $\varepsilon$-LDP and ultra-mixing. The results are applied to bound Markov-chain mixing times, offering a non-linear contraction perspective on $L^{\alpha}$-norm convergence and providing complementary tools to classical norms for assessing convergence rates in finite alphabets.

Abstract

This work explores properties of Strong Data-Processing constants for Rényi Divergences. Parallels are made with the well-studied $\varphi$-Divergences, and it is shown that the order $α$ of Rényi Divergences dictates whether certain properties of the contraction of $\varphi$-Divergences are mirrored or not. In particular, we demonstrate that when $α>1$, the contraction properties can deviate quite strikingly from those of $\varphi$-Divergences. We also uncover specific characteristics of contraction for the $\infty$-Rényi Divergence and relate it to $\varepsilon$-Local Differential Privacy. The results are then applied to bound the speed of convergence of Markov chains, where we argue that the contraction of Rényi Divergences offers a new perspective on the contraction of $L^α$-norms commonly studied in the literature.

Figures (4)

• Figure 1: Comparison between the SDPI constants $\eta_{\alpha}(\mu, K)$, $\eta_{\mathcal{H}_\alpha}(\mu, K)$ and $\eta_{\chi^2}(\mu, K)$ for $\alpha\in(0, 10]$, where $K=0.50.50.10.9$ and $\mu = 0.90.1$. The two regimes outlined in \ref{['corollary:sdpi_ub_lb_hellinger']} can be observed depending on the range of $\alpha$.
• Figure 2: Plot of the bounds in \ref{['prop:bounds_on_renyi_infty_sdpi_by_tv']} for $K=\mathrm{BSC(\varepsilon)}$ and $\mu=0.50.5$ together with $\eta_{\chi^2}(\mu, K)$ for comparison.
• Figure 3: Comparison of distribution-independent SDPI constants for $K=\mathrm{BSC(\varepsilon)}$.
• Figure 4: Plot of the bounds on $\left\|\frac{\mathrm{d}\nu K^n}{\mathrm{d}\pi}-1\right\|_{L^2(\pi)}^2$ from \ref{['eq:mcmt_non_linear_sdpi_alpha_2', 'eq:mcmt_linear_sdpi_alpha_2']} as a function of $n$. The Markov chain corresponds to ${K=\mathrm{BSC}(\varepsilon)\otimes \mathrm{BSC}(\varepsilon)\otimes \mathrm{BSC}(\varepsilon)}$ with $\varepsilon=10^{-2}$. The stationary distribution $\pi$ is uniform and $\nu$ is a Dirac mass. The SDPI constants appearing in the bounds are evaluated numerically. Additional details can be found in \ref{['appendix:mixing_time_experiment']}.

Theorems & Definitions (20)

• definition 1
• definition 2
• definition 3
• proposition 1
• remark 1
• theorem 1
• proposition 2
• theorem 2
• example 1
• theorem 3