Universal Slepian-Wolf coding for individual sequences
Neri Merhav
TL;DR
The paper addresses universal Slepian-Wolf coding for two deterministic sequences with unknown joint statistics, introducing finite-state encoders and a universal decoder based on Lempel-Ziv complexities. It derives a coding theorem characterizing the achievable rate region in terms of $\rho(\boldmath x|\boldmath y)$, $\rho(\boldmath y|\boldmath x)$, and $\rho(\boldmath x,\boldmath y)$, and proves an asymptotic chain-rule like relation $\rho(\boldmath x,\boldmath y)=\rho(\boldmath x|\boldmath y)+\rho(\boldmath y|\boldmath x)$ via $\rho^+=\rho^-=\rho$. The paper then presents a universal incremental SW coding scheme, extending Draper’s approach to the individual-sequence setting with a low-rate feedback link to adapt rates to the blockwise LZ compressibilities, and shows that this scheme universally attains the described rate region. By bridging finite-state universal decoding with LZ-based complexity measures, the work extends Slepian-Wolf coding to unknown, non-stochastic sources and provides a practical, adaptive framework for joint decoding of deterministic sequences.
Abstract
We establish a coding theorem and a matching converse theorem for separate encodings and joint decoding of individual sequences using finite-state machines. The achievable rate region is characterized in terms of the Lempel-Ziv (LZ) complexities, the conditional LZ complexities and the joint LZ complexity of the two source sequences. An important feature that is needed to this end, which may be interesting on its own right, is a certain asymptotic form of a chain rule for LZ complexities, which we establish in this work. The main emphasis in the achievability scheme is on the universal decoder and its properties. We then show that the achievable rate region is universally attainable by a modified version of Draper's universal incremental Slepian-Wolf (SW) coding scheme, provided that there exists a low-rate reliable feedback link.