Rényi Common Information for Doubly Symmetric Binary Sources

TL;DR

This paper determines the Rényi common information for doubly symmetric binary sources (DSBS) across orders including $(1,\infty)$ and negative orders. It provides analytic expressions for the DSBS Rényi CI, proves a complete characterization for $\alpha\in[0,1]$ where the value reduces to Wyner's common information, and derives DSBS-specific expressions and tight bounds for $\alpha\in(1,\infty]$ and $\alpha\in[-\infty,0)$. A key contribution is the entropy-splitting lemma and the analytic expression of relaxed Wyner's common information, which enable upper and lower bounds that match in the DSBS case and reveal a numerically observed phase transition around a threshold $\epsilon_0\approx0.0551$. The work links Rényi CI to the nonnegative rank and mutual-information regions, and it provides a conjectural phase-transition phenomenon for negative orders, with practical implications for distributed source synthesis and information-theoretic rank concepts. The results generalize the understanding of common randomness requirements and offer tractable, DSBS-specific formulas for a broad range of Rényi orders.

Abstract

In this note, we provide analytic expressions for the Rényi common information of orders in $(1,\infty)$ for the doubly symmetric binary source (DSBS). Until now, analytic expressions for the Rényi common information of all orders in $[0,\infty]$ have been completely known for this source. We also consider the Rényi common information of all orders in $[-\infty,0)$ and evaluate it for the DSBS. We provide a sufficient condition under which the Rényi common information of such orders coincides with Wyner's common information for the DSBS. Based on numerical analysis, we conjecture that there is a certain phase transition as the crossover probability increasing for the Rényi common information of negative orders for the DSBS. Our proofs are based on a lemma on splitting of the entropy and the analytic expression of relaxed Wyner's common information.

Figures (2)

• Figure 1: Distributed source synthesis problem, where $M_{n}$ is a random variable uniformly distributed over $\mathcal{M}_{n}:=\{1,\ldots,2^{nR}\}$.
• Figure 2: Illustrations of the Rényi common informations for the DSBS with crossover probability $\epsilon=0.3$. Wyner common information and the Rényi common information of order $\infty$ respectively form a lower and an upper bound on the Rényi common informations of all orders except order $0$. For order $0$, the Rényi common information is zero.

Theorems & Definitions (10)

• Theorem 1: Rényi Common Informations
• Theorem 2: Rényi Common Information for DSBS
• Conjecture 1
• Proposition 1: Relaxed Wyner's Common Information for DSBS
• Claim 1
• Lemma 1: Splitting of Entropy
• Lemma 2
• Proposition 2
• Lemma 3
• Proposition 3