Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Entropic Bounds on the Average Length of Codes with a Space

Roberto Bruno, Ugo Vaccaro

TL;DR

The paper addresses prefix-free coding with a space symbol restricted to appear only at the end of codewords. It develops a linear-time method to construct almost-optimal space-ending codes, showing their average length is within one unit of the minimum, by exploiting a fundamental link to $k$-ary one-to-one codes and deriving entropy-based bounds. It provides both lower and upper bounds on the average length of optimal space-ending codes in terms of the source entropy $H_k({f p})$ and the alphabet size $k$, including refinements that depend on the largest symbol probability $p_1$. The work clarifies the trade-offs introduced by the space constraint, demonstrates that the constraint’s impact diminishes as $k$ grows, and outlines open questions for achieving truly optimal codes via possibly dynamic-programming approaches across the space-end constraint.

Abstract

We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that their average length differs from the minimum possible by at most one. We obtain our results by uncovering a relation between our class of codes and the class of one-to-one codes. Additionally, we derive upper and lower bounds to the average length of optimal prefix-free codes with a space in terms of the source entropy.

Entropic Bounds on the Average Length of Codes with a Space

TL;DR

The paper addresses prefix-free coding with a space symbol restricted to appear only at the end of codewords. It develops a linear-time method to construct almost-optimal space-ending codes, showing their average length is within one unit of the minimum, by exploiting a fundamental link to -ary one-to-one codes and deriving entropy-based bounds. It provides both lower and upper bounds on the average length of optimal space-ending codes in terms of the source entropy and the alphabet size , including refinements that depend on the largest symbol probability . The work clarifies the trade-offs introduced by the space constraint, demonstrates that the constraint’s impact diminishes as grows, and outlines open questions for achieving truly optimal codes via possibly dynamic-programming approaches across the space-end constraint.

Abstract

We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that their average length differs from the minimum possible by at most one. We obtain our results by uncovering a relation between our class of codes and the class of one-to-one codes. Additionally, we derive upper and lower bounds to the average length of optimal prefix-free codes with a space in terms of the source entropy.
Paper Structure (5 sections, 12 theorems, 20 equations)

This paper contains 5 sections, 12 theorems, 20 equations.

Table of Contents

  1. Introduction
  2. Relations between One-to-One Codes and Prefix Codes Ending with a Space
  3. Lower Bounds on the Average Length
  4. Upper Bounds on the Average Length
  5. Concluding Remarks

Key Result

Lemma 1

Let $S=\{s_1,\ldots,s_n\}$ be the set of source symbols and ${\mathbf{p}}=(p_1,\ldots , p_n)$, $p_1\geq\ldots \geq p_n>0$, be a probability distribution on $S$. Let $\{0,\dots,k-1\}$ be the $k\geq 2$-ary code alphabet. We can construct a prefix-free code $C:S\longmapsto\{0,\dots,k-1\}^+ \cup \{0,\do where $L_+$ is the average length of an optimal one-to-one code $D:S\longmapsto \{0,\dots,k-1\}^+$

Theorems & Definitions (13)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Theorem 1
  • Theorem 2
  • Lemma 7
  • Lemma 8
  • ...and 3 more