Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Measuring the Redundancy of Information from a Source Failure Perspective

Jesse Milzman

TL;DR

This paper defines a new measure of the redundancy of information from a fault tolerance perspective based upon the presupposition that redundant information is robust to individual source failures, and demonstrates that this new measure satisfies the common PID axioms from (Williams 2010).

Abstract

In this paper, we define a new measure of the redundancy of information from a fault tolerance perspective. The partial information decomposition (PID) emerged last decade as a framework for decomposing the multi-source mutual information $I(T;X_1, ..., X_n)$ into atoms of redundant, synergistic, and unique information. It built upon the notion of redundancy/synergy from McGill's interaction information (McGill 1954). Separately, the redundancy of system components has served as a principle of fault tolerant engineering, for sensing, routing, and control applications. Here, redundancy is understood as the level of duplication necessary for the fault tolerant performance of a system. With these two perspectives in mind, we propose a new PID-based measure of redundancy $I_{\text{ft}}$, based upon the presupposition that redundant information is robust to individual source failures. We demonstrate that this new measure satisfies the common PID axioms from (Williams 2010). In order to do so, we establish an order-reversing correspondence between collections of source-fallible instantiations of a system, on the one hand, and the PID lattice from (Williams 2010), on the other.

Measuring the Redundancy of Information from a Source Failure Perspective

TL;DR

This paper defines a new measure of the redundancy of information from a fault tolerance perspective based upon the presupposition that redundant information is robust to individual source failures, and demonstrates that this new measure satisfies the common PID axioms from (Williams 2010).

Abstract

In this paper, we define a new measure of the redundancy of information from a fault tolerance perspective. The partial information decomposition (PID) emerged last decade as a framework for decomposing the multi-source mutual information into atoms of redundant, synergistic, and unique information. It built upon the notion of redundancy/synergy from McGill's interaction information (McGill 1954). Separately, the redundancy of system components has served as a principle of fault tolerant engineering, for sensing, routing, and control applications. Here, redundancy is understood as the level of duplication necessary for the fault tolerant performance of a system. With these two perspectives in mind, we propose a new PID-based measure of redundancy , based upon the presupposition that redundant information is robust to individual source failures. We demonstrate that this new measure satisfies the common PID axioms from (Williams 2010). In order to do so, we establish an order-reversing correspondence between collections of source-fallible instantiations of a system, on the one hand, and the PID lattice from (Williams 2010), on the other.
Paper Structure (4 sections, 5 theorems, 21 equations, 1 table)

This paper contains 4 sections, 5 theorems, 21 equations, 1 table.

Table of Contents

  1. Introduction
  2. Motivating Example
  3. Partial Information Decomposition
  4. A Measure of Redundant Information in Source-Fallible Systems

Key Result

Proposition 1

For any base system $\mathfrak{s}$ and subsystem $\mathfrak{s}_I \subset [n]$, there is an injective embedding $g_I: \mathfrak{F}_I \hookrightarrow \mathfrak{F}$, given by This embedding is information preserving, in the sense that

Theorems & Definitions (14)

  • Definition 1: Base System
  • Definition 2: Source-Fallible System
  • Definition 3
  • Definition 4
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3: Self-Redundancy Axiom
  • proof
  • ...and 4 more