Probabilistic analysis of arithmetic coding showing its robustness
Hosam M. Mahmoud, Hans J. Rivertz
TL;DR
This work analyzes the probabilistic performance of arithmetic coding under a Bernoulli($p$) binary source by deriving a functional equation for the bivariate moment generating function of the final interval endpoints $(X_n,Y_n)$. Through a method of moments and a family of moment matrices $W_m$, the authors show that the final codeword converges in distribution to $Uniform[0,1]$ regardless of $p$, with $p$ only influencing the rate of convergence via terms like $(1-2pq)^n$. The analysis establishes concentration of the interval length and provides exact and asymptotic expressions for low-order moments, demonstrating exponential convergence and almost-sure uniform limits. The key takeaway is the intrinsic robustness of arithmetic coding across the entire class of binary sources considered, offering theoretical justification for practical underflow resilience and decoding fidelity.
Abstract
We probabilistically analyze the performance of the arithmetic coding algorithm under a probability model for binary data in which a message is received by a coder from a source emitting independent equally distributed bits, with 1 occurring with probability $p\in(0,1)$ and 0 occurring with probability $1-p$. We establish a functional equation for the bivariate moment generating function for the two ends of the final interval delivered by the algorithm. Via the method of moments, we show that the transmitted message converges in distribution to the standard continuous uniform random variable on the interval [0,1]. It is remarkable that the limiting distribution is the same for all $p$, indicating robustness in the performance of arithmetic coding across an entire family of bit distributions. The nuance with $p$ appears only in the rate of convergence.