Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Structural Impossibility of Antichain-Lattice Partial Information Decomposition

Aobo Lyu, Andrew Clark, Netanel Raviv

Abstract

Partial Information Decomposition (PID) represents multivariate mutual information via antichain-lattice that aims to specify which source groups can recover which informational components of a target. For three or more sources, widely desired PID axioms become mutually incompatible. This is often treated as an axiomatic tuning issue. This paper argues that the obstruction is representational, rooted in the antichain indexing itself, so that purely axiomatic adjustments within an antichain-lattice structure cannot resolve it in general. We first introduce System Information Decomposition (SID) for the special target-free three-variable setting, obtaining a self-consistent entropy decomposition with an operational redundancy definition. More fundamentally, we then show that for general multivariate PID, there is no universal rule that recovers the decomposed mutual information from the antichain-indexed information atoms. In particular, two systems can share identical atoms regardless of any axioms while having different mutual information. These results reveal the limits of antichain-lattice and motivate relation-based foundations for multivariate information measures.

Structural Impossibility of Antichain-Lattice Partial Information Decomposition

Abstract

Partial Information Decomposition (PID) represents multivariate mutual information via antichain-lattice that aims to specify which source groups can recover which informational components of a target. For three or more sources, widely desired PID axioms become mutually incompatible. This is often treated as an axiomatic tuning issue. This paper argues that the obstruction is representational, rooted in the antichain indexing itself, so that purely axiomatic adjustments within an antichain-lattice structure cannot resolve it in general. We first introduce System Information Decomposition (SID) for the special target-free three-variable setting, obtaining a self-consistent entropy decomposition with an operational redundancy definition. More fundamentally, we then show that for general multivariate PID, there is no universal rule that recovers the decomposed mutual information from the antichain-indexed information atoms. In particular, two systems can share identical atoms regardless of any axioms while having different mutual information. These results reveal the limits of antichain-lattice and motivate relation-based foundations for multivariate information measures.

Paper Structure

This paper contains 20 sections, 10 theorems, 54 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Partial Information Decomposition
  3. PID framework and redundancy lattice
  4. Inconsistency for three or more sources
  5. System Information Decomposition
  6. Structural Limitations of Antichain-Lattice
  7. Discussion
  8. Comparison between SID and two sources PID
  9. Proofs of Main Results
  10. Proof of Lemma \ref{['lemma: counter example']}
  11. Derivation of SID Axiom \ref{['sid axiom:mutual constrains']}
  12. Proof of Lemma \ref{['lemma:UniqueUpToOneAtom']}
  13. Proof of Lemma \ref{['lemma: satisfaction']}
  14. Proof of Lemma \ref{['lem:canonical-atom-assignment']}
  15. Proof of Lemma \ref{['lemma:groundtruth-pair']}
  16. ...and 5 more sections

Key Result

Lemma 1

rauh2014reconsidering Let $S_1$ and $S_2$ be two independent $\operatorname{Bernoulli}(1/2)$ variables, let $S_3$ be their exclusive OR (XOR), and let $T=(S_1,S_2,S_3)$. Then, any PID measure satisfies PID Axioms PID Axiom: Commutativity, pid axiom: Monotonicity, pid axiom: Self-redundancy, Property where $I(T;\mathbf{S})=2$, but three non-zero atoms have value $1$. $\blacktriangleleft$$\blacktria

Figures (4)

  • Figure 1: The structure of PID with two source variables, i.e., \ref{["equ:Information Atoms' relationship_1"]}\ref{["equ:Information Atoms' relationship_2"]}.
  • Figure 2: Comparison between SID and three-source PID. (A) Three-variable SID. (B) Three-source PID, where the antichains in bold contain at least one singleton source, whose structure is consistent with SID.
  • Figure 3: Three-source systems $(\hat{S}_1,\hat{S}_2,\hat{S}_3,\hat{T})$ and $(\tilde{S}_1,\tilde{S}_2,\tilde{S}_3,\tilde{T})$ constructed from latent bits $x_1$ to $x_9$.
  • Figure 4: Comparison between SID and two-source PID.

Theorems & Definitions (27)

  • Definition 1: PID Redundancy Lattice
  • Definition 2: Partial Information Decomposition Framework
  • Lemma 1
  • Definition 3: SID Half Lattice
  • Definition 4: System Information Decomposition Framework
  • Proposition 1: Symmetric synergy from SID Axiom \ref{['sid axiom:mutual constrains']}
  • proof
  • Remark 1
  • Lemma 2
  • Definition 5: Operational Definition of Redundancy
  • ...and 17 more