The mathematical landscape of partial information decomposition: A comprehensive review of properties and measures

Abstract

Partial Information Decomposition (PID) has become one of the most prominent information-theoretic frameworks for describing the structure and quality of information in complex systems. Despite its widespread utility, there exists no unique solution constraining precisely how a PID should be constructed, leading to a multiverse of different formalisms with different mathematical commitments. In this work, we provide a comprehensive overview of the mathematical landscape of PID. By integrating existing PID measures into a common language, we systematically examine all major approaches to the PID framework that have emerged so far, determining for each measure whether or not each known property holds. In addition, we derive a web of all known theorems mapping the relationships and incompatibilities between these properties, before also revealing some novel interdependency results. In doing so, we chart a brief history of the framework, promote a unified perspective for its discussions, and offer a path towards both theoretical refinement and informed empirical applications for the future of this powerful method.

Figures (4)

• Figure 1: Hierarchical organisation of PID measures induced by properties.a) Number of measures satisfying each mathematical property (left), and number of properties satisfied by each measure (right), from Table \ref{['tab:PID_properties']}. (SR), (EI) and (S$_{\text{0}}$) are satisfied by all measures, whereas (S$_{\text{1}}$) is currently satisfied by no measures. $I_\cap^{\text{BROJA}}\xspace$ satisfies the most properties (16), whereas $I_\cap^{\text{CCS}}\xspace$ satisfies the least (5). b) Hierarchical clustering of the PID measures based on mathematical properties. $I_\cap^{\text{CCS}}\xspace$, $I_\cap^{\text{PM}}\xspace$ and $I_\cap^{\text{SX}}\xspace$ clearly emerge as a distant community of measures likely due to their unique lack of possession of (GP). $I_\cap^{\text{MES}}\xspace$ and $I_\cap^{\text{do}}\xspace$ occupy another cluster in-between this and the main community of measures, likely due to the absence of (M$_{\text{0}}$), (M$_{\text{1}}$), (LB), which they share with $I_\cap^{\text{CCS}}\xspace$, $I_\cap^{\text{PM}}\xspace$ and $I_\cap^{\text{SX}}\xspace$. The main cluster comprises $I_\cap^{\text{RDR}}\xspace$, $I_\cap^{\text{min}}\xspace$, $I_\cap^{\text{MMI}}\xspace$, $I^{\text{red}}_\cap\xspace$, $I_\cap^{\text{BROJA}}\xspace$, $I_\cap^{\delta}\xspace$, $I_\cap^{\text{IG}}\xspace$, $I_\cap^{\text{RR}}\xspace$, $I_\cap^{\text{RAV}}\xspace$, $I_\cap^{\text{DEP}}\xspace$, and $I_\cap^{\text{CT}}\xspace$. A particularly compact subgroup is unsurprisingly formed by $I_\cap^{\text{RDR}}\xspace$, $I_\cap^{\text{min}}\xspace$ and $I_\cap^{\text{MMI}}\xspace$ due to their (lack of) performance in the TBC ((ID) and (IID)), as well as their possession of (LP$_{\text{0}}$). Similarly, $I^{\text{red}}_\cap\xspace$, $I_\cap^{\text{BROJA}}\xspace$, $I_\cap^{\delta}\xspace$ form a tight subgroup as a consequence of their possession of (BP), ($\boldsymbol{\ast}$) but not (LP$_{\text{0}}$).
• Figure 2: Formal relationships among PID properties.a) Directed hypergraph representing logical implications of PID properties from Table \ref{['tab:PID_implications']}, hyperedges are coloured by order of implication (upper). In- and out-degree distributions for property implications reveal that (M$_{\text{0}}$) and (SR) imply the most properties (4), whereas (TE) and (GP) are implied by the most properties (3). b) Undirected hypergraph represented logical incompatibilities from Table \ref{['tab:PID_incompatibilities']}. Degree distributions for property incompatibilities reveal that despite implying the most properties, (SR) is also incompatible with the most properties (4), tied with (EI).
• Figure 3: Partial information decomposition lattices for a)$n=2$ and b)$n=3$ sources.
• Figure 4: Temporal timeline of the introduction of PID measures, properties, and their relationships.$^*$ refers to the inequality: $I_\cap(X_1,X_2;f(X_1,X_2))\le I_\cap(X_1;X_2)$ (see Prop. \ref{['prop:ID-TM--ineq']}).

Theorems & Definitions (119)

• Lemma 1
• Lemma 2
• Lemma 3
• Lemma 4
• Proposition 1: griffith2014intersection
• proof
• Proposition 2: williams2010nonnegative
• proof
• Proposition 3: williams2010nonnegative
• proof