Characterizations of Conditional Mutual Independence: Equivalence and Implication

TL;DR

This work studies conditional mutual independence (CMI) for a finite set of discrete random variables, resolving two core problems: when two CMIs $K$ and $K'$ are equivalent and when $K$ implies $K'$, via a canonical form ${\rm can}(K)$ derived from the pure form ${\rm pur}(K)$. It proves that $K \sim K'$ if and only if ${\rm can}({\rm pur}(K))={\rm can}({\rm pur}(K'))$, and that $K$ implies $K'$ precisely when $K'$ is a sub-CMI of $K$, using the residual $K''=R^{K'}_{K}$ and a structured definition of sub-CMI. The results unify equivalence and implication of CMIs within an entropic framework and introduce canonical/pure representations to enable exact, algorithmic characterizations. The findings have implications for probabilistic reasoning and graphical models, with potential extensions to general (non-discrete) distributions and deeper connections to information-theoretic inequalities.

Abstract

Conditional independence, and more generally conditional mutual independence, are central notions in probability theory. In their general forms, they include functional dependence as a special case. In this paper, we tackle two fundamental problems related to conditional mutual independence. Let $K$ and $K'$ be two conditional mutual independncies (CMIs) defined on a finite set of discrete random variables. We have obtained a necessary and sufficient condition for i) $K$ is equivalent to $K'$; ii) $K$ implies $K'$. These characterizations are in terms of a canonical form introduced for conditional mutual independence.

Theorems & Definitions (121)

• Definition 1.1: Conditional Independence
• Definition 1.2: Conditional Mutual Independence
• Definition 1.3: Conditional Pairwise Independence
• Definition 1.4
• Definition 2.1
• Definition 2.2
• Proposition 2.1: Chain Rule for Conditional Entropy
• Proposition 2.2: Chain Rule for Conditional Mutual Information
• Corollary 2.1
• proof