Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Whole Is Less than the Sum of Parts: Subsystem Inconsistency in Partial Information Decomposition

Aobo Lyu, Andrew Clark, Netanel Raviv

TL;DR

The paper challenges the PID framework by showing that the whole information content need not equal the sum of PID atoms in multivariate systems, unveiling WESP violations through a three-source counterexample. It introduces System Information Decomposition (SID) for three-variable systems, defining a half-lattice and SI-atoms that resolve WESP by reconfiguring how redundancy and synergy are summed, and links SID to an extended Gács–Körner redundancy construction. It also proves that for four or more variables, no fixed antichain-based summation can universally fix WESP, revealing a fundamental limitation of lattice-based decompositions. Collectively, the work advocates moving beyond traditional antichain lattices toward frameworks that can capture higher-order, holistic synergy, potentially via entropy-centric decompositions rather than mutual-information partitions.

Abstract

Partial Information Decomposition (PID) was proposed by Williams and Beer in 2010 as a tool for analyzing fine-grained interactions between multiple random variables, and has since found numerous applications ranging from neuroscience to privacy. However, a unified theoretical framework remains elusive due to key conceptual and technical challenges. We identify and illustrate a crucial problem: PID violates the set-theoretic principle that the whole equals the sum of its parts (WESP). Through a counterexample in a three-variable system, we demonstrate how such violations naturally arise, revealing a fundamental limitation of current lattice-based PID frameworks. To address this issue, we introduce a new axiomatic framework, termed System Information Decomposition (SID), specifically tailored for three-variable systems. SID resolves the WESP violation by redefining the summation rules of decomposed information atoms based on synergistic relationships. However, we further show that for systems with four or more variables, no partial summation approach within the existing lattice-based structures can fully eliminate WESP inconsistencies. Our results thus highlight the inherent inadequacy of (antichain) lattice-based decompositions for general multivariate systems.

The Whole Is Less than the Sum of Parts: Subsystem Inconsistency in Partial Information Decomposition

TL;DR

The paper challenges the PID framework by showing that the whole information content need not equal the sum of PID atoms in multivariate systems, unveiling WESP violations through a three-source counterexample. It introduces System Information Decomposition (SID) for three-variable systems, defining a half-lattice and SI-atoms that resolve WESP by reconfiguring how redundancy and synergy are summed, and links SID to an extended Gács–Körner redundancy construction. It also proves that for four or more variables, no fixed antichain-based summation can universally fix WESP, revealing a fundamental limitation of lattice-based decompositions. Collectively, the work advocates moving beyond traditional antichain lattices toward frameworks that can capture higher-order, holistic synergy, potentially via entropy-centric decompositions rather than mutual-information partitions.

Abstract

Partial Information Decomposition (PID) was proposed by Williams and Beer in 2010 as a tool for analyzing fine-grained interactions between multiple random variables, and has since found numerous applications ranging from neuroscience to privacy. However, a unified theoretical framework remains elusive due to key conceptual and technical challenges. We identify and illustrate a crucial problem: PID violates the set-theoretic principle that the whole equals the sum of its parts (WESP). Through a counterexample in a three-variable system, we demonstrate how such violations naturally arise, revealing a fundamental limitation of current lattice-based PID frameworks. To address this issue, we introduce a new axiomatic framework, termed System Information Decomposition (SID), specifically tailored for three-variable systems. SID resolves the WESP violation by redefining the summation rules of decomposed information atoms based on synergistic relationships. However, we further show that for systems with four or more variables, no partial summation approach within the existing lattice-based structures can fully eliminate WESP inconsistencies. Our results thus highlight the inherent inadequacy of (antichain) lattice-based decompositions for general multivariate systems.
Paper Structure (13 sections, 8 theorems, 64 equations, 5 figures)

This paper contains 13 sections, 8 theorems, 64 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Partial Information Decomposition
  3. Three-variable System Information Decomposition
  4. Limitations of Anti-chains based Information Decomposition
  5. System 1 ($\hat{S}_1,\hat{S}_2,\hat{S}_3,\hat{T}$)
  6. System 2 ($\bar{S}_1,\bar{S}_2,\bar{S}_3,\bar{T}$)
  7. Discussion
  8. Comparison between SID and two sources PID
  9. Proofs of Main Results
  10. Proof of Lemma \ref{['lemma: counter example']}
  11. Proof of Lemma \ref{['lemma:UniqueUpToOneAtom']}
  12. Proof of Lemma \ref{['lemma: satisfaction']}
  13. Proof of Lemma \ref{['lemma:NoUniversalSubset']}

Key Result

Lemma 1

For $\mathbf{A},\mathbf{B},\mathbf{C}\subseteq \mathbf{S}$ such that $\mathbf{C}\subseteq\mathbf{A}\cap \mathbf{B}$, let $\Pi$ from Definition pid def:PIDF that satisfying PID Axiom pid axiom:mutual constrains, we have that

Figures (5)

  • Figure 1: The structure of PID with two source variables, i.e., \ref{["equ:Information Atoms' relationship_1"]}\ref{["equ:Information Atoms' relationship_2"]}.
  • Figure 2: The structure of PID with 3 source variables.
  • Figure 3: Comparison between SID and three sources PID. (A) Three-variable SID. (B) Three-sources PID, where the antichains in bold contain at least one singleton source, whose structure is consistent with SID.
  • Figure 4: Construction of $(\hat{S}_1,\hat{S}_2,\hat{S}_3,\hat{T})$ and ($\bar{S}_1,\bar{S}_2,\bar{S}_3,\bar{T}$).
  • Figure 5: Comparison between SID and two sources PID.

Theorems & Definitions (20)

  • Definition 1: PID Redundancy Lattice
  • Definition 2: Partial Information Decomposition Framework
  • Lemma 1: Subsystem Consistency
  • Lemma 2: Nonnegativity
  • proof
  • Remark 1
  • Lemma 3
  • Definition 3: SID Half Lattice
  • Definition 4: System Information Decomposition Framework
  • Lemma 4
  • ...and 10 more