Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Distributionally Robust Approach to Shannon Limits using the Wasserstein Distance

Vikrant Malik, Taylan Kargin, Victoria Kostina, Babak Hassibi

TL;DR

The paper addresses robustness of Shannon limits when source or noise distributions are uncertain within a $W_2$-ambiguity set around a nominal model. It shows that, for Gaussian centers, the worst-case rate-distortion and capacity-cost functions are attained by Gaussian distributions, and provides convex SDP/LMI formulations that yield closed-form results in the scalar case. By linking $W_2$ geometry through the Gelbrich bound and Gaussian saddle-point properties, the authors derive explicit dual and primal characterizations: $R_{P_ extcircled{0}}(D,r)=\sup_{BW(\Sigma,\Sigma_0)\le r} R_{\text{N}(0,\Sigma)}(D)$ and $C_{P_ extcircled{0}}(B,r)=\inf_{BW(\Sigma,\Sigma_0)\le r} C_{\text{N}(0,\Sigma)}(B)$, with scalar forms $R$ and $C$ reducing to $\tfrac{1}{2}\log^{+}((\sigma_0+r)^2/D)$ and $\tfrac{1}{2}\log(1+B/(\sigma_0+r)^2)$, respectively. These results interpolate between known nominal limits and fully robust limits as $r$ grows, providing insight into how Shannon limits evolve under distributional uncertainty. The framework opens doors to extensions to other transport distances, multiterminal settings, and causal coding, with practical impact for robust data compression and communication with uncertain statistics.

Abstract

We consider the rate-distortion function for lossy source compression, as well as the channel capacity for error correction, through the lens of distributional robustness. We assume that the distribution of the source or of the additive channel noise is unknown and lies within a Wasserstein-2 ambiguity set of a given radius centered around a specified nominal distribution, and we look for the worst-case asymptotically optimal coding rate over such an ambiguity set. Varying the radius of the ambiguity set allows us to interpolate between the worst-case and stochastic scenarios using probabilistic tools. Our problem setting fits into the paradigm of compound source / channel models introduced by Sakrison and Blackwell, respectively. This paper shows that if the nominal distribution is Gaussian, then so is the worst-case source / noise distribution, and the compound rate-distortion / channel capacity functions admit convex formulations with Linear Matrix Inequality (LMI) constraints. These formulations yield simple closed-form expressions in the scalar case, offering insights into the behavior of Shannon limits with the changing radius of the Wasserstein-2 ambiguity set.

A Distributionally Robust Approach to Shannon Limits using the Wasserstein Distance

TL;DR

The paper addresses robustness of Shannon limits when source or noise distributions are uncertain within a -ambiguity set around a nominal model. It shows that, for Gaussian centers, the worst-case rate-distortion and capacity-cost functions are attained by Gaussian distributions, and provides convex SDP/LMI formulations that yield closed-form results in the scalar case. By linking geometry through the Gelbrich bound and Gaussian saddle-point properties, the authors derive explicit dual and primal characterizations: and , with scalar forms and reducing to and , respectively. These results interpolate between known nominal limits and fully robust limits as grows, providing insight into how Shannon limits evolve under distributional uncertainty. The framework opens doors to extensions to other transport distances, multiterminal settings, and causal coding, with practical impact for robust data compression and communication with uncertain statistics.

Abstract

We consider the rate-distortion function for lossy source compression, as well as the channel capacity for error correction, through the lens of distributional robustness. We assume that the distribution of the source or of the additive channel noise is unknown and lies within a Wasserstein-2 ambiguity set of a given radius centered around a specified nominal distribution, and we look for the worst-case asymptotically optimal coding rate over such an ambiguity set. Varying the radius of the ambiguity set allows us to interpolate between the worst-case and stochastic scenarios using probabilistic tools. Our problem setting fits into the paradigm of compound source / channel models introduced by Sakrison and Blackwell, respectively. This paper shows that if the nominal distribution is Gaussian, then so is the worst-case source / noise distribution, and the compound rate-distortion / channel capacity functions admit convex formulations with Linear Matrix Inequality (LMI) constraints. These formulations yield simple closed-form expressions in the scalar case, offering insights into the behavior of Shannon limits with the changing radius of the Wasserstein-2 ambiguity set.
Paper Structure (15 sections, 8 theorems, 42 equations, 1 figure)

This paper contains 15 sections, 8 theorems, 42 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Literature context
  3. Contributions
  4. Main results
  5. Source Compression
  6. Channel Coding
  7. Proof of Theorem \ref{['th:main']}
  8. Useful Results
  9. Proof of Theorem \ref{['th:main']}: upper bound
  10. Proof of Theorem \ref{['th:main']}: lower bound
  11. Conclusion
  12. Proof of Lemma \ref{['th:worst_case_rdf_vector']}
  13. Proof of Theorem \ref{['th:main_cap']}
  14. Converse
  15. Achievability

Key Result

Theorem 2.1

The compound RDF eq:RDg with a Gaussian center $P_\circ = \mathcal{N} \left( 0, \Sigma_\circ \right)$ is given as, where the function $R_{P_\circ}(D, r)$ is achieved by a Gaussian $P_X$.

Figures (1)

  • Figure 1: The scalar compound capacity for $P_{\circ} = \mathcal{N}(0,1)$ for $r = 0$. The scalar compound capacity for $r = 0$ is the Shannon capacity for $P_\circ$ and for $r > 0$ is the Shannon capacity for $P_Z = \mathcal{N}(0, (1 + r)^2)$. As the radius of the ambiguity set increases, the required input power $B$ to achieve the same capacity increases.

Theorems & Definitions (8)

  • Theorem 2.1: Compound RDF for $\operatorname{W_2}$ ambiguity set
  • Lemma 2.2: Strong duality of the compound RDF
  • Theorem 2.3: Compound capacity-cost function for $\operatorname{W_2}$ ambiguity set
  • Lemma 2.4
  • Lemma 3.1: Gaussian is the hardest to encode, kolmogorov1956shannon
  • Lemma A.1: Gaussian input maximizes the mutual information in an AWGN channel
  • Lemma B.1: Capacity of an AWGN channel
  • Lemma B.2: Gaussian noise minimizes the channel capacity of an additive channel