Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information Inequalities for Five Random Variables

E. P. Csirmaz, L. Csirmaz

Abstract

The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown. We utilize a variant of the Maximum Entropy Method to obtain five-variable non_shannon entropy inequalities, which delimit the five-variable entropy region. This method adds copies of some of the random variables in generations. A significant reduction in computational complexity, achieved through theoretical considerations and by harnessing the inherent symmetries, allowed us to calculate all five-variable non-Shannon inequalities provided by the first nine generations. Based on the results, we define two infinite collections of such inequalities and prove them to be entropy inequalities. We investigate downward-closed subsets of non-negative lattice points that parameterize these collections, and based on this, we develop an algorithm to enumerate all extremal inequalities. The discovered set of entropy inequalities is conjectured to characterize the applied method completely.

Information Inequalities for Five Random Variables

Abstract

The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown. We utilize a variant of the Maximum Entropy Method to obtain five-variable non_shannon entropy inequalities, which delimit the five-variable entropy region. This method adds copies of some of the random variables in generations. A significant reduction in computational complexity, achieved through theoretical considerations and by harnessing the inherent symmetries, allowed us to calculate all five-variable non-Shannon inequalities provided by the first nine generations. Based on the results, we define two infinite collections of such inequalities and prove them to be entropy inequalities. We investigate downward-closed subsets of non-negative lattice points that parameterize these collections, and based on this, we develop an algorithm to enumerate all extremal inequalities. The discovered set of entropy inequalities is conjectured to characterize the applied method completely.
Paper Structure (19 sections, 15 theorems, 74 equations, 1 figure, 6 tables)

This paper contains 19 sections, 15 theorems, 74 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The Maximum Entropy Method
  4. What to Compute? How to Compute?
  5. Tight and Modular Parts
  6. Symmetry
  7. No New Inequality
  8. Problem Parameters
  9. Computation
  10. Case I
  11. Case II
  12. Case III
  13. Experimental Information Inequalities
  14. Case I
  15. Case II
  16. ...and 4 more sections

Key Result

Lemma 1

In the distribution with maximum total entropy, the subsets $Y_1,\dots, Y_n$ and $Z_1,\dots, Z_m$ are completely conditionally independent over $X$.

Figures (1)

  • Figure 1: Nearest horizontal and vertical boundary segments of the gray region are marked by the arrows. Red circles indicate points to be added and subtracted.

Theorems & Definitions (47)

  • Lemma 1
  • proof
  • Definition 1
  • Claim 2
  • proof
  • Lemma 3
  • Claim 4
  • proof
  • Claim 5
  • proof
  • ...and 37 more