Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Proving Information Inequalities by Gaussian Elimination

Laigang Guo, Raymond W. Yeung, Xiao-Shan Gao

TL;DR

This work introduces a symbolic, Gaussian-elimination–driven framework to prove information inequalities and identities under linear constraints on Shannon measures, offering an alternative to LP-based methods like ITIP. By homogenizing objectives, ordering entropies, and applying dimension-reduction techniques with Gauss-Jordan elimination, the method often proves inequalities without solving any LP and, when needed, reduces the problem to a smaller LP solvable by specialized procedures. The authors formalize two procedures—Procedure I for inequalities and Procedure II for identities—together with targeted preprocessing steps (QuickIMP-RED) and heuristic searches that transform the problem into conic decompositions in terms of nonnegative variables $a_i$. They demonstrate the approach on Dougherty–Freiling–Zeger’s problem and Tian’s problem, achieving substantial reductions in variable counts and LP size with competitive or superior performance compared to direct LP methods and existing tools. The work also provides enhancements and detailed mappings (Final-1) linking entropy expressions to algebraic variables, suggesting a practical, scalable path for proving complex information-theoretic inequalities.

Abstract

The proof of information inequalities and identities under linear constraints on the information measures is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and implemented, which are all based on solving a linear program (LP). In this paper, we develop a method with symbolic computation. Compared with the known methods, our approach can completely avoids the use of linear programming which may cause numerical errors. Our procedures are also more efficient computationally.

Proving Information Inequalities by Gaussian Elimination

TL;DR

This work introduces a symbolic, Gaussian-elimination–driven framework to prove information inequalities and identities under linear constraints on Shannon measures, offering an alternative to LP-based methods like ITIP. By homogenizing objectives, ordering entropies, and applying dimension-reduction techniques with Gauss-Jordan elimination, the method often proves inequalities without solving any LP and, when needed, reduces the problem to a smaller LP solvable by specialized procedures. The authors formalize two procedures—Procedure I for inequalities and Procedure II for identities—together with targeted preprocessing steps (QuickIMP-RED) and heuristic searches that transform the problem into conic decompositions in terms of nonnegative variables . They demonstrate the approach on Dougherty–Freiling–Zeger’s problem and Tian’s problem, achieving substantial reductions in variable counts and LP size with competitive or superior performance compared to direct LP methods and existing tools. The work also provides enhancements and detailed mappings (Final-1) linking entropy expressions to algebraic variables, suggesting a practical, scalable path for proving complex information-theoretic inequalities.

Abstract

The proof of information inequalities and identities under linear constraints on the information measures is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and implemented, which are all based on solving a linear program (LP). In this paper, we develop a method with symbolic computation. Compared with the known methods, our approach can completely avoids the use of linear programming which may cause numerical errors. Our procedures are also more efficient computationally.
Paper Structure (12 sections, 6 theorems, 25 equations, 2 tables, 1 algorithm)

This paper contains 12 sections, 6 theorems, 25 equations, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Information inequality preliminaries
  3. Algorithms for homogeneous linear inequlities
  4. Procedures for proving information inequality and identity
  5. Procedure I: Proving Information Inequalities
  6. Procedure II: Proving Information Identities
  7. Two Applications
  8. Dougherty-Freiling-Zeger's Problem
  9. Tian's Problem
  10. Conclusion and discussion
  11. Two enhancements of Algorithm \ref{['QuickIMP-RED']}
  12. Formulas in \ref{['Final-1']}

Key Result

Theorem 2.1

Yeung1997 Any Shannon's information measure can be expressed as a conic combination of the following two elemental forms of Shannon's information measures: i) $H(X_i|X_{\mathcal{N}_n-\{i\}})$ ii) $I(X_i;X_j|X_K)$, where $i\neq j$ and $K\subseteq \mathcal{N}_n-\{i,j\}$.

Theorems & Definitions (33)

  • Theorem 2.1
  • Example 2.1
  • Theorem 2.2
  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Definition 3.4
  • Definition 3.5
  • Lemma 3.1
  • Lemma 3.2
  • ...and 23 more