Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Measure of Synergy based on Union Information

André F. C. Gomes, Mário A. T. Figueiredo

TL;DR

This paper tackles the challenge of decomposing information in the partial information decomposition (PID) framework by introducing a new union-information measure based on a conditional-independence (CI) channel perspective. The authors define $I_ ext{cup}^ ext{CI}$ and the associated synergy $S^ ext{CI}$, derive a multivariate extension, and provide an operational interpretation: synergy quantifies information beyond what is captured when sources are assumed conditionally independent given the target. They extend the Williams–Beer axioms to union information, analyze the measure against existing proposals, and demonstrate its desirable properties (nonnegativity, monotonicity) along with limitations (e.g., some target-monotonicity properties may fail). The work places CI-based union information in relation to prior measures (e.g., $I_ ext{cup}^ ext{d}$, $I_ ext{cup}^ ext{VK}$, $I_ ext{cup}^ ext{BROJA}$) and connects to recent PID developments like James 2018 unique, highlighting practical computability and potential for broad applications. The paper also provides code for implementation and outlines future directions, including extensions to continuous variables and integration with the dit package, underscoring its practical impact for applications in neuroscience, cryptography, and network analysis.

Abstract

The partial information decomposition (PID) framework is concerned with decomposing the information that a set of (two or more) random variables (the sources) has about another variable (the target) into three types of information: unique, redundant, and synergistic. Classical information theory alone does not provide a unique way to decompose information in this manner and additional assumptions have to be made. One often overlooked way to achieve this decomposition is using a so-called measure of union information - which quantifies the information that is present in at least one of the sources - from which a synergy measure stems. In this paper, we introduce a new measure of union information based on adopting a communication channel perspective, compare it with existing measures, and study some of its properties. We also include a comprehensive critical review of characterizations of union information and synergy measures that have been proposed in the literature.

A Measure of Synergy based on Union Information

TL;DR

This paper tackles the challenge of decomposing information in the partial information decomposition (PID) framework by introducing a new union-information measure based on a conditional-independence (CI) channel perspective. The authors define and the associated synergy , derive a multivariate extension, and provide an operational interpretation: synergy quantifies information beyond what is captured when sources are assumed conditionally independent given the target. They extend the Williams–Beer axioms to union information, analyze the measure against existing proposals, and demonstrate its desirable properties (nonnegativity, monotonicity) along with limitations (e.g., some target-monotonicity properties may fail). The work places CI-based union information in relation to prior measures (e.g., , , ) and connects to recent PID developments like James 2018 unique, highlighting practical computability and potential for broad applications. The paper also provides code for implementation and outlines future directions, including extensions to continuous variables and integration with the dit package, underscoring its practical impact for applications in neuroscience, cryptography, and network analysis.

Abstract

The partial information decomposition (PID) framework is concerned with decomposing the information that a set of (two or more) random variables (the sources) has about another variable (the target) into three types of information: unique, redundant, and synergistic. Classical information theory alone does not provide a unique way to decompose information in this manner and additional assumptions have to be made. One often overlooked way to achieve this decomposition is using a so-called measure of union information - which quantifies the information that is present in at least one of the sources - from which a synergy measure stems. In this paper, we introduce a new measure of union information based on adopting a communication channel perspective, compare it with existing measures, and study some of its properties. We also include a comprehensive critical review of characterizations of union information and synergy measures that have been proposed in the literature.
Paper Structure (22 sections, 3 theorems, 25 equations, 3 figures, 10 tables)

This paper contains 22 sections, 3 theorems, 25 equations, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Background
  3. PID Based of Channel Orders
  4. Motivation for a New PID
  5. A New Measure of Union Information
  6. Motivation and Bivariate Definition
  7. Operational Interpretation
  8. General (Multivariate) Definition
  9. Properties of Measures of Union Information and Synergy
  10. Extension of the Williams--Beer Axioms for Measures of Union Information
  11. Review of Suggested Properties: griffith2014quantifying
  12. Stronger Monotonicity
  13. Target Monotonicity
  14. Review of Suggested Properties: quax2017quantifying
  15. Review of Suggested Properties: rosas2020operational
  16. ...and 7 more sections

Key Result

Theorem 1

$I^\text{CI}_\cup$ satisfies the Williams--Beer axioms for measures of union information given in Definition WB.

Figures (3)

  • Figure 1: (a) Assuming faithfulness pearl2009causality, this is the only three-variable directed acyclic graph (DAG) that satisfies $Y_1 \perp Y_2$ and $Y_1 \not\perp Y_2 | T$, in general pearl2009causality. (b) The DAG that is "implied" by the perspective of $I_\cap^\text{d}$. (c) A DAG that can generate the XOR distribution, but does not satisfy the dependencies implied by $T = Y_1 \text{ xor } Y_2$. In fact, any DAG that is in the same Markov equivalence class as (c) can generate the XOR distribution (or any other joint distribution), but none satisfy the earlier dependencies, assuming faithfulness.
  • Figure 2: Trivariate distribution lattices and their respective ordering of sources. Left (a): synergy lattice chicharro2017synergy. Right (b): union information semi-lattice gutknecht2023babel.
  • Figure 3: Computation of $S^{CI}$ and $S^{d}$ as functions of $r=p(T=0|Y=(0,0))$ for the distribution presented in Table \ref{['adaptedxorv2']}. As we showed for this distribution, $S^{CI}$ is not a convex function of $r$.

Theorems & Definitions (10)

  • Example 3.1
  • Example 3.2
  • Definition 1
  • Definition 2
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof