Novel Inconsistency Results for Partial Information Decomposition

TL;DR

The paper demonstrates fundamental incompatibilities when importing classical information-theoretic properties into Partial Information Decomposition (PID) using a mereological framework. It proves that non-negativity, re-encoding invariance, and a target chain rule cannot all hold simultaneously in general, and strengthens earlier results by showing incompatibilities without relying on the identity property. Using the XOR-Source-Copy Gate with three sources, it derives contradictions for multiple property combinations and discusses reformulations across PID-inducing concepts. The work highlights essential trade-offs for PID definitions and argues that some properties (notably REI) are particularly resistant to relaxation, while others may be traded off depending on the application, thereby clarifying long-standing debates in the PID literature.

Abstract

Partial Information Decomposition (PID) seeks to disentangle how information about a target variable is distributed across multiple sources, separating redundant, unique, and synergistic contributions. Despite extensive theoretical development and applications across diverse fields, the search for a unique, universally accepted solution remains elusive, with numerous competing proposals offering different decompositions. A promising but underutilized strategy for making progress is to establish inconsistency results, proofs that certain combinations of intuitively appealing axioms cannot be simultaneously satisfied. Such results clarify the landscape of possibilities and force us to recognize where fundamental choices must be made. In this work, we leverage the recently developed mereological approach to PID to establish novel inconsistency results with far-reaching implications. Our main theorem demonstrates that three cornerstone properties of classical information theory, namely non-negativity, the chain rule, and invariance under invertible transformations, become mutually incompatible when extended to the PID setting. This result reveals that any PID framework must sacrifice at least one property that seems fundamental to information theory itself. Additionally, we strengthen the classical result of Rauh et al., which showed that non-negativity, the identity property, and the Williams and Beer axioms cannot coexist.

Figures (2)

• Figure 1: Left: redundancy lattice $(\mathcal{A}_3, \preceq)$. Right: Information diagrams of the corresponding redundancy terms. Clearly visible is the nested structure of redundant information.
• Figure 2: Illustration of the degree of redundancy $r(\alpha)$ of information atoms for $n=3$. The degree of redundancy measures how many individual mutual information terms $I(S_i;T)$ an atom contributes to, and hence, how often it appears in the first term in Equation \ref{['eq:rsi_degr_red']}. The atoms with $r(\alpha)=0$ are synergistic in the sense that they cannot be obtained from any individual information source. These only appear in the second term of Equation \ref{['eq:rsi_degr_red']} and hence end up with a minus sign in Equation \ref{['eq:rsi_degr_red_2']}. The atoms with $r(\alpha)=1$ appear exactly once in both terms and hence cancel. The atoms with $r(\alpha)\geq 2$, one might say the genuinely redundant ones, end up with a plus sign and multiplicity $r(\alpha)-1$ in Equation \ref{['eq:rsi_degr_red_2']}.

Theorems & Definitions (32)

• Definition 1: Parthood Distribution
• Definition 2: Partial Information Decomposition
• Definition 3: Conditional Information Atoms
• Definition 4: Redundant Information
• Proposition 1
• proof
• Definition 5: $\mathcal{C}$-Information
• Definition 6: Union Information
• Definition 7: Weak Synergy
• Definition 8: Vulnerable Information