Optimal Redundancy in Exact Channel Synthesis
Sharang M. Sriramu, Aaron B. Wagner
TL;DR
This work resolves the second-order redundancy for exact channel synthesis in the i.i.d. setting by distinguishing singular and non-singular channels. It introduces a two-stage rejection-sampling scheme that separately simulates a quantized log-likelihood ratio and the conditional channel, achieving a redundancy of $ rac{1}{2} rac{ ext{log} n}{n}$ for non-singular channels and sublogarithmic redundancy for singular ones, with a tight bound for full-support DMCs. A matching converse shows the $(1/2)$ factor is unavoidable for non-singular channels with full support, clarifying the fundamental limits of redundancy in exact channel synthesis. These results have practical significance for implementing channel-synthesis-based compression in high dimensions, including DNN-based approaches where codebook size and training cost scale rapidly with dimension.
Abstract
We consider the redundancy of the exact channel synthesis problem under an i.i.d. assumption. Existing results provide an upper bound on the unnormalized redundancy that is logarithmic in the block length. We show, via an improved scheme, that the logarithmic term can be halved for most channels and eliminated for all others. For full-support discrete memoryless channels, we show that this is the best possible.