Explicit Formula for Partial Information Decomposition

TL;DR

The paper addresses the lack of an explicit PID formula that satisfies Williams-Beer axioms for a three-variable system. It introduces a do-operation–based construction to define the unique information atom, $Un(X\to Z|Y)$, via $I(X';Z|Y)$ with $X'$ built from a do-operation on $(X,Z)$, and derives the redundant and synergistic atoms from fundamental relationships. The authors prove that the resulting decomposition satisfies the PID axioms—commutativity, monotonicity/self-redundancy, nonnegativity—and key properties such as additivity and continuity, providing a coherent, operational framework. This work yields a rigorous, explicit PID formula with potential applications across neuroscience, privacy, and causality by enabling precise attribution of information to unique, shared, and synergistic contributions.

Abstract

Mutual information between two random variables is a well-studied notion, whose understanding is fairly complete. Mutual information between one random variable and a pair of other random variables, however, is a far more involved notion. Specifically, Shannon's mutual information does not capture fine-grained interactions between those three variables, resulting in limited insights in complex systems. To capture these fine-grained interactions, in 2010 Williams and Beer proposed to decompose this mutual information to information atoms, called unique, redundant, and synergistic, and proposed several operational axioms that these atoms must satisfy. In spite of numerous efforts, a general formula which satisfies these axioms has yet to be found. Inspired by Judea Pearl's do-calculus, we resolve this open problem by introducing the do-operation, an operation over the variable system which sets a certain marginal to a desired value, which is distinct from any existing approaches. Using this operation, we provide the first explicit formula for calculating the information atoms so that Williams and Beer's axioms are satisfied, as well as additional properties from subsequent studies in the field.

Figures (1)

• Figure 1: A pictorial representation of Partial Information Decomposition \ref{['eqn:basicRelation']}, where $I((X,Y);Z)$ is decomposed to its finer information atoms, the synergistic $\operatorname{Syn}(X,Y\to Z)$ (also called "complementary"), the redundant $\operatorname{Red}(X,Y\to Z)$ (also called "shared"), and the two directional unique components $\operatorname{Un}(X\to Z|Y)$ and $\operatorname{Un}(Y\to Z|X)$. The summation of the redundant atom and one of the unique atoms must be equal to the corresponding mutual information, as described in Eq. \ref{["equ:Information Atoms' relationship_2"]}.

Theorems & Definitions (27)

• Definition 1: Unique Information
• Definition 2: Redundant Information
• Definition 3: Synergistic Information
• Definition 4: Do-operation
• Definition 5: Unique Information, equivalent definition
• Lemma 1
• Corollary 1
• Corollary 2
• Lemma 2
• Lemma 3: Commutativity of Redundant Information