Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shared Information for a Markov Chain on a Tree

Sagnik Bhattacharya, Prakash Narayan

TL;DR

This work studies shared information $SI(X_1, \ldots,X_m)$ for a Markov chain on a tree (MCT) and provides a direct, tree-structured analysis to obtain an explicit $SI$ characterization. The authors prove a global Markov property for MCTs based on graph separation, and show equivalence to the standard MCT definitions even when the joint pmf is not strictly positive. When the joint distribution is unknown but the tree is known, they formulate a correlated bandits approach that identifies the weakest edge by estimating edge mutual informations via the empirical mutual information estimator $I_{\mathsf{EMI}}^{(n)}$, with rigorous bounds on estimation error and sample complexity. The key result is that $SI(X_\mathcal{M}) = \min_{(i,j) \in \mathcal{E}} I(X_i;X_j)$, enabling a simple edge-based computation, and the analysis provides a concrete procedure for data driven SI estimation on trees with provable performance guarantees. This yields practical tools for multiterminal secrecy, clustering, and distributed randomness generation in tree-structured settings.

Abstract

Shared information is a measure of mutual dependence among multiple jointly distributed random variables with finite alphabets. For a Markov chain on a tree with a given joint distribution, we give a new proof of an explicit characterization of shared information. The Markov chain on a tree is shown to possess a global Markov property based on graph separation; this property plays a key role in our proofs. When the underlying joint distribution is not known, we exploit the special form of this characterization to provide a multiarmed bandit algorithm for estimating shared information, and analyze its error performance.

Shared Information for a Markov Chain on a Tree

TL;DR

This work studies shared information for a Markov chain on a tree (MCT) and provides a direct, tree-structured analysis to obtain an explicit characterization. The authors prove a global Markov property for MCTs based on graph separation, and show equivalence to the standard MCT definitions even when the joint pmf is not strictly positive. When the joint distribution is unknown but the tree is known, they formulate a correlated bandits approach that identifies the weakest edge by estimating edge mutual informations via the empirical mutual information estimator , with rigorous bounds on estimation error and sample complexity. The key result is that , enabling a simple edge-based computation, and the analysis provides a concrete procedure for data driven SI estimation on trees with provable performance guarantees. This yields practical tools for multiterminal secrecy, clustering, and distributed randomness generation in tree-structured settings.

Abstract

Shared information is a measure of mutual dependence among multiple jointly distributed random variables with finite alphabets. For a Markov chain on a tree with a given joint distribution, we give a new proof of an explicit characterization of shared information. The Markov chain on a tree is shown to possess a global Markov property based on graph separation; this property plays a key role in our proofs. When the underlying joint distribution is not known, we exploit the special form of this characterization to provide a multiarmed bandit algorithm for estimating shared information, and analyze its error performance.
Paper Structure (11 sections, 13 theorems, 94 equations, 4 figures)

This paper contains 11 sections, 13 theorems, 94 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Properties of an MCT
  4. Shared Information for a Markov Chain on a Tree
  5. Estimating $\mathop{\mathrm{SI}}\nolimits(X_\mathcal{M})$ for an MCT
  6. Preliminaries
  7. Bandit algorithm for estimating $\mathop{\mathrm{SI}}\nolimits(X_\mathcal{M})$
  8. Closing Remarks
  9. Proof of \ref{['lem:mrf-mi']}
  10. Proof of \ref{['lem:mct-local-prop']} and \ref{['th:g-g']}
  11. Proof of \ref{['lem:tech-1']}

Key Result

Lemma 1

For the MCT $\mathcal{G} = (\mathcal{M}, \mathcal{E})$, for every $(i,j) \in \mathcal{E}$, i.e.,

Figures (4)

  • Figure 1: Notation for a Markov chain on a tree.
  • Figure 2: Example 2 of an MCT.
  • Figure 3: Schematic for proof of \ref{['eq:mct-mi-proof-claim-2']}.
  • Figure 4: Schematic for the proof of \ref{['lem:mct-local-prop']}.

Theorems & Definitions (42)

  • Definition 1: Shared information
  • Example 1
  • Remark 1
  • Definition 2: Markov Chain on a Tree
  • Remark 2
  • Example 2
  • Lemma 1
  • proof
  • Definition 3: Agglomerated Tree
  • Lemma 2
  • ...and 32 more