Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Orders between channels and implications for partial information decomposition

André F. C. Gomes, Mário A. T. Figueiredo

TL;DR

This work addresses how to enrich partial information decomposition (PID) by basing intersection information on channel preorders. Building on Kolchinsky's axiomatic approach, it introduces three new II measures corresponding to the less noisy, more capable, and degradation/supermodularity preorders, each defined as I(Q;T) maximized over admissible Q with Q ⪯ Y_i under the respective preorder. The authors establish that any preorder satisfying Kolchinsky's axioms yields a valid II measure obeying the Williams–Beer axioms, and they show that when a degradation relation exists among sources the new measures coincide, while in the absence of degradation they yield novel, finer decompositions. They also formulate optimization problems for computing these measures and relate them to existing PID constructions, arguing that their approach broadens the landscape of redundancy measures while maintaining a solid theoretical foundation. The work points to future avenues, including conditions for ds ordering, continuity, broader entropy frameworks, and potential quantum extensions, highlighting both theoretical and practical implications for information-flow analysis.

Abstract

The partial information decomposition (PID) framework is concerned with decomposing the information that a set of random variables has with respect to a target variable into three types of components: redundant, synergistic, and unique. Classical information theory alone does not provide a unique way to decompose information in this manner and additional assumptions have to be made. Recently, Kolchinsky proposed a new general axiomatic approach to obtain measures of redundant information, based on choosing an order relation between information sources (equivalently, order between communication channels). In this paper, we exploit this approach to introduce three new measures of redundant information (and the resulting decompositions) based on well-known preorders between channels, thus contributing to the enrichment of the PID landscape. We relate the new decompositions to existing ones, study some of their properties, and provide examples illustrating their novelty. As a side result, we prove that any preorder that satisfies Kolchinsky's axioms yields a decomposition that meets the axioms originally introduced by Williams and Beer when they first propose the PID.

Orders between channels and implications for partial information decomposition

TL;DR

This work addresses how to enrich partial information decomposition (PID) by basing intersection information on channel preorders. Building on Kolchinsky's axiomatic approach, it introduces three new II measures corresponding to the less noisy, more capable, and degradation/supermodularity preorders, each defined as I(Q;T) maximized over admissible Q with Q ⪯ Y_i under the respective preorder. The authors establish that any preorder satisfying Kolchinsky's axioms yields a valid II measure obeying the Williams–Beer axioms, and they show that when a degradation relation exists among sources the new measures coincide, while in the absence of degradation they yield novel, finer decompositions. They also formulate optimization problems for computing these measures and relate them to existing PID constructions, arguing that their approach broadens the landscape of redundancy measures while maintaining a solid theoretical foundation. The work points to future avenues, including conditions for ds ordering, continuity, broader entropy frameworks, and potential quantum extensions, highlighting both theoretical and practical implications for information-flow analysis.

Abstract

The partial information decomposition (PID) framework is concerned with decomposing the information that a set of random variables has with respect to a target variable into three types of components: redundant, synergistic, and unique. Classical information theory alone does not provide a unique way to decompose information in this manner and additional assumptions have to be made. Recently, Kolchinsky proposed a new general axiomatic approach to obtain measures of redundant information, based on choosing an order relation between information sources (equivalently, order between communication channels). In this paper, we exploit this approach to introduce three new measures of redundant information (and the resulting decompositions) based on well-known preorders between channels, thus contributing to the enrichment of the PID landscape. We relate the new decompositions to existing ones, study some of their properties, and provide examples illustrating their novelty. As a side result, we prove that any preorder that satisfies Kolchinsky's axioms yields a decomposition that meets the axioms originally introduced by Williams and Beer when they first propose the PID.
Paper Structure (14 sections, 3 theorems, 29 equations, 1 figure, 4 tables)

This paper contains 14 sections, 3 theorems, 29 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Kolchinsky's Axioms and Intersection Information
  3. Channels and the Degradation/Blackwell Preorder
  4. Other Preorders and Corresponding II Measures
  5. The "Less Noisy" Preorder
  6. The "More Capable" preorder
  7. The "Degradation/Supermodularity" preorder
  8. Relations Between Preorders
  9. Optimization Problems
  10. The "Less Noisy" Preorder
  11. The "More Capable" Preorder
  12. The "Degradation/Supermodularity" Preorder
  13. Relation with Existing PID Measures
  14. Conclusion and future work

Key Result

Theorem 1

Let $\preceq$ be some preorder that satisfies Kolchinsky's axioms and define its corresponding II measure as in II. Then the corresponding II measure satisfies the Williams-Beer axioms.

Figures (1)

  • Figure 1: Implications satisfied by the preorders. The reverse implications do not hold in general.

Theorems & Definitions (14)

  • Definition 1
  • Theorem 1
  • proof
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Theorem 2
  • ...and 4 more