Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Source Coding with Free Bits and the Multi-Way Number Partitioning Problem

Niloufar Ahmadypour, Amin Gohari

TL;DR

This paper addresses a two-channel source coding problem with a free and a costly channel, reducing the encoding task to partitioning the source alphabet into k subsets and encoding within subsets. It establishes that a variant of Huffman coding, the Early Stopping Huffman algorithm, is optimal for the compression objective, which minimizes the conditional entropy H(X|A). The authors then connect this problem to the classical number partitioning problem, introducing a compression objective and an alpha-divergence objective, along with bounds and a principle of optimality for entropy-based and related objectives. The work advances a bridge between information theory and combinatorial partitioning, yielding an efficient O(n log n) method for exact solutions in the compression setting and provable approximation bounds for the broader alpha-divergence family, with potential implications for resource allocation and fair division in distributed systems.

Abstract

We introduce a new variant of variable-length source coding for sending a source over two parallel channels, one of which is costly and the other free. We give a complete solution to this problem. Next, we relate the problem to the number partitioning problem, which is the task of dividing a given list of numbers into a pre-specified number of subsets such that the sum of the numbers in each subset is as nearly equal as possible. We introduce two new objective functions for this problem and show that an adapted version of the Huffman coding algorithm (with a runtime of $\mathcal{O}(n \log n)$ for input size $n$) produces the optimal solution for one objective function, and a nearly optimal solution for the other objective function.

Source Coding with Free Bits and the Multi-Way Number Partitioning Problem

TL;DR

This paper addresses a two-channel source coding problem with a free and a costly channel, reducing the encoding task to partitioning the source alphabet into k subsets and encoding within subsets. It establishes that a variant of Huffman coding, the Early Stopping Huffman algorithm, is optimal for the compression objective, which minimizes the conditional entropy H(X|A). The authors then connect this problem to the classical number partitioning problem, introducing a compression objective and an alpha-divergence objective, along with bounds and a principle of optimality for entropy-based and related objectives. The work advances a bridge between information theory and combinatorial partitioning, yielding an efficient O(n log n) method for exact solutions in the compression setting and provable approximation bounds for the broader alpha-divergence family, with potential implications for resource allocation and fair division in distributed systems.

Abstract

We introduce a new variant of variable-length source coding for sending a source over two parallel channels, one of which is costly and the other free. We give a complete solution to this problem. Next, we relate the problem to the number partitioning problem, which is the task of dividing a given list of numbers into a pre-specified number of subsets such that the sum of the numbers in each subset is as nearly equal as possible. We introduce two new objective functions for this problem and show that an adapted version of the Huffman coding algorithm (with a runtime of for input size ) produces the optimal solution for one objective function, and a nearly optimal solution for the other objective function.

Paper Structure

This paper contains 10 sections, 6 theorems, 41 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Number partitioning problem
  3. New objective functions for the number partition problem
  4. Preliminaries
  5. The Source Coding Problem
  6. Number Partition Problem
  7. Compression objective function
  8. Principle of optimality
  9. $\alpha$-divergence objective function
  10. Conclusion

Key Result

Lemma 4

Suppose that $\boldsymbol{P}_n(\rho)$ for $n\geq 2$ and $\rho > 1$ is the set of all probability mass vectors $\boldsymbol{p}$ (sorted in descending order) of size $n$ with $\dfrac{p_1}{p_n} \leq \rho$, and $\boldsymbol{u}_n$ is the Uniform probability mass vector, then for each $\boldsymbol{p} \in and if $\alpha =1$ we get where the RHS of eq74 is the limit $\alpha \to 1$ of the RHS of eq73.

Figures (1)

  • Figure 1: Transmission of a discrete memoryless source over two parallel noiseless channels: a free channel conveying the subset index and a costly binary channel conveying the remaining codeword bits.

Theorems & Definitions (15)

  • Definition 1
  • Definition 2
  • Definition 3: The Rényi divergence renyi
  • Lemma 4
  • proof
  • Theorem 5
  • Lemma 6
  • proof : Proof of Lemma \ref{['implemma']}
  • proof : Proof of Theorem \ref{['mainthm']}
  • Theorem 7
  • ...and 5 more