Data-Driven Estimation of Capacity Upper Bounds

TL;DR

This work addresses estimating upper bounds on the capacity of memoryless channels with unknown laws and continuous outputs by exploiting the dual representation $C = \min_{\gamma \geq 0} [F(\gamma) + \gamma P]$ with $F(\gamma) = \min_{q_Y} \max_{x} [ D(f_{Y|X=x} || q_Y) - \gamma c(x) ]$. It introduces a data-driven divergence estimator based on the DV representation, augmented by conditioning on the channel input via a neural statistics network, and couples this with a neural distribution transformer (NDT) to adaptively learn $q_Y$. An alternating optimization procedure trains the divergence estimator and the reference distribution, enabling upper-bound estimation without requiring a differentiable channel model. Empirical results on AWGN, Optical Intensity, and NLPN channels show the estimated bounds converge toward channel capacity or tight lower bounds, demonstrating the method’s potential for model-free capacity estimation while acknowledging bias and discretization caveats at finite samples.

Abstract

We consider the problem of estimating an upper bound on the capacity of a memoryless channel with unknown channel law and continuous output alphabet. A novel data-driven algorithm is proposed that exploits the dual representation of capacity where the maximization over the input distribution is replaced with a minimization over a reference distribution on the channel output. To efficiently compute the required divergence maximization between the conditional channel and the reference distribution, we use a modified mutual information neural estimator that takes the channel input as an additional parameter. We numerically evaluate our approach on different memoryless channels and show empirically that the estimated upper bounds closely converge either to the channel capacity or to best-known lower bounds.

Figures (5)

• Figure 1: Accuracy and generalization ability of the trained divergence estimator \ref{['eq:modified_mine']}, where $\mathcal{X}_t = \{-1, -0.8, \ldots, +1\}$.
• Figure 2: Two approaches for implementing the neural distribution transformer (NDT) that generates samples from the reference distribution $q_Y$.
• Figure 3: Results for the AWGN channel.
• Figure 4: Results for the OI channel with $\alpha=0.4$ (cf. Lapidoth2009)
• Figure 5: Results for the NLPN channel.