Generalized Forms of the Kraft Inequality for Finite-State Encoders

TL;DR

This work extends Kraft’s inequality to information-lossless finite-state encoders by introducing the Kraft matrix $K$ whose entries encode state- and symbol-level transition costs, and by showing that a necessary condition for ILFS encoding is a spectral radius bound $\rho(K)\le 1$. In the irreducible case, the analysis yields stronger, uniform bounds via Perron–Frobenius theory and even linear-growth-free bounds on $K^{n}$, providing tighter converse results and links to empirical entropy and Lempel–Ziv complexity. The framework is further extended to settings with side information through the joint spectral radius of a finite set of Kraft matrices, and to lossy compression by bounding growth via distortion-induced factors $B_{\ell}$, connecting to rate-distortion quantities $\Phi(D)$. Overall, the paper offers an exact, state-level characterization of feasibility for ILFS encoders, improves on previous GKI formulations, and supplies tools for analyzing compression and prediction under finite-memory constraints. The results have implications for universal coding, prediction, and the design of memory-limited encoders, enabling verifiable, structure-preserving bounds on achievable rates and growth of Kraft sums.

Abstract

We derive a few extended versions of the Kraft inequality for information lossless finite-state encoders. The main basic contribution is in defining a notion of a Kraft matrix and in establishing the fact that a necessary condition for information losslessness of a finite-state encoder is that none of the eigenvalues of this matrix have modulus larger than unity, or equivalently, the generalized Kraft inequality asserts that the spectral radius of the Kraft matrix cannot exceed one. For the important special case where the FS encoder is irreducible, we derive several equivalent forms of this inequality, which are based on well known formulas for spectral radius. It also turns out that in the irreducible case, Kraft sums are bounded by a constant, independent of the block length, and thus cannot grow even in any subexponential rate. Finally, two extensions are outlined - one concerns the case of side information available to both encoder and decoder, and the other is for lossy compression.

Figures (2)

• Figure 1: State transition diagram of the encoder in Example 1. The various state transitions are labeled in a form $x/y$, where $x$ denotes the input and $y=f(z,x)$ denotes the output.
• Figure 2: Auxiliary predictive encoder and decoder. The upper block diagram depicts the encoder that losslessly compresses the prediction error signal, $z_n$, which is the difference between the input signal, $x_n$, and its prediction, $\hat{x}_n$ obtained using a FS predictor. The lower block diagram stands for the corresponding decoder.

Theorems & Definitions (11)

• Theorem 1
• proof
• Example 1
• Theorem 2
• proof
• Definition 1
• Definition 2
• Theorem 3
• proof
• Proposition 1