Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Algebraic Representations of Entropy and Fixed-Parity Information Quantities

Keenan J. A. Down, Pedro A. M. Mediano

Abstract

Many information-theoretic quantities have corresponding representations in terms of sets. The prevailing signed measure space for characterising entropy, the $I$-measure of Yeung, is occasionally unable to discern between qualitatively distinct systems. In previous work, we presented a refinement of this signed measure space and demonstrated its capability to represent many quantities, which we called logarithmically decomposable quantities. In the present work we demonstrate that this framework has natural algebraic behaviour which can be expressed in terms of ideals (characterised here as upper-sets), and we show that this behaviour allows us to make various counting arguments and characterise many fixed-parity information quantity expressions. As an application, we give an algebraic proof that the only completely synergistic system of three finite variables $X$, $Y$ and $Z = f(X,Y)$ is the XOR gate.

Algebraic Representations of Entropy and Fixed-Parity Information Quantities

Abstract

Many information-theoretic quantities have corresponding representations in terms of sets. The prevailing signed measure space for characterising entropy, the -measure of Yeung, is occasionally unable to discern between qualitatively distinct systems. In previous work, we presented a refinement of this signed measure space and demonstrated its capability to represent many quantities, which we called logarithmically decomposable quantities. In the present work we demonstrate that this framework has natural algebraic behaviour which can be expressed in terms of ideals (characterised here as upper-sets), and we show that this behaviour allows us to make various counting arguments and characterise many fixed-parity information quantity expressions. As an application, we give an algebraic proof that the only completely synergistic system of three finite variables , and is the XOR gate.
Paper Structure (10 sections, 19 theorems, 46 equations, 2 figures)

This paper contains 10 sections, 19 theorems, 46 equations, 2 figures.

Table of Contents

  1. Introduction
  2. An Algebraic Perspective on Entropy
  3. Representing information quantities inside $\Delta \Omega$
  4. Ideals in $\Delta\Omega$
  5. Representation of quantities with ideals
  6. Properties of the Measure on Ideals
  7. Fixed-Parity Systems
  8. Conclusion
  9. Background on the measure
  10. Proofs for Results

Key Result

Theorem 4

Let $R$ be a region on an $I$-diagram of variables $X_1,\ldots, X_r$ with Yeung's $I$-measure. In particular, $R$ is given by some set-theoretic expression in terms of the set variables $\tilde{X}_1, \ldots, \tilde{X}_r$ under some combination of unions, intersections and set differences. Making the to obtain an expression $\Delta R$ in terms of the $\Delta X_i$, we have That is, the interior los

Figures (2)

  • Figure 1: An outcome space $\Omega = \{1,2,3\}$ and two variables $X$ and $Y$ defined over $\Omega$. In this case, the intersection of the contents $\Delta X \cap \Delta Y$ is given by the ideal $\langle 12\rangle$. That is to say, $I(X;Y) = \mu(\langle 12 \rangle)$. If the mutual information could be represented by a partition in this case, we would get something like the above intersection. This is, of course, impossible in the language of partitions, but valid in ideals.
  • Figure 2: An $I$-diagram demonstrating the entropy structure for the OR gate. The shaded region corresponds to the ideal $\langle 14, 123\rangle$. Note that in this case the degree of the generators is bounded above by 3, as we have the intersection of 3 variables as per theorem \ref{['THEOREM_GeneratorDegreeBound']}.

Theorems & Definitions (50)

  • Example 1
  • Definition 2
  • Definition 3
  • Theorem 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Definition 8
  • Proposition 10
  • Remark 11
  • ...and 40 more