A Pair of Bayesian Network Structures has Undecidable Conditional Independencies

TL;DR

The paper addresses whether the conditional independencies implied by two Bayesian network structures can be algorithmically decided. It proves that, unlike the single-structure case where d-separation provides a complete, efficient decision procedure, the CI implications of two structures are undecidable, even for singleton conditioning sets. The authors develop new constructions that impose functional dependencies on sufficient statistics and employ majorization, culminating in a reduction from an undecidable CI-implication problem to the two-network setting. This result implies there is no general algorithm to combine two BN structures into a single structure or data structure that can answer all CI queries. The work also shows, under a consistency assumption for ZFC, there exist two BN structures whose implication for a CI is unprovable in ZFC, highlighting profound limits on qualitative reasoning with BN structures.

Abstract

Given a Bayesian network structure (directed acyclic graph), the celebrated d-separation algorithm efficiently determines whether the network structure implies a given conditional independence relation. We show that this changes drastically when we consider two Bayesian network structures instead. It is undecidable to determine whether two given network structures imply a given conditional independency, that is, whether every collection of random variables satisfying both network structures must also satisfy the conditional independency. Although the approximate combination of two Bayesian networks is a well-studied topic, our result shows that it is fundamentally impossible to accurately combine the knowledge of two Bayesian network structures, in the sense that no algorithm can tell what conditional independencies are implied by the two network structures. We can also explicitly construct two Bayesian network structures, such that whether they imply a certain conditional independency is unprovable in the ZFC set theory, assuming ZFC is consistent.

Figures (2)

• Figure 1: Two examples of the problem of combining the knowledge of two Bayesian network structures. In the first example on top, if we know that the random variables $W,X,Y,Z$ satisfy Network 1 (which implies the CIs $W\perp\!\!\!\perp(Y,Z)|X$ and $(W,X)\perp\!\!\!\perp Z|X$) and Network 2 (which implies the CI $(X,W)\perp\!\!\!\perp Y$), then this is equivalent to $W,X,Y,Z$ satisfying Network 3 (which implies the CI $(X,W)\perp\!\!\!\perp(Y,Z)$). This CI implied by Network 3 is not implied by Network 1 or 2, and hence we cannot simply take the union of the set of CIs implied by Network 1 and the set of CIs implied by Network 2 to obtain the combined set of CIs. The combination might not even be a Bayesian network, as shown in the second example below. These observations have been noted in del2003qualitative. This paper shows that generally, the combined set of CIs cannot even be computed.
• Figure 2: An illustration of the two networks when $n=4$, $k=3$, $\mathcal{C}_{1}=\{1,2\}$, $\mathcal{C}_{1}=\{2,3\}$, $\mathcal{A}_{1}=\{1,3\}$, $b_{1}=2$, $\mathcal{A}_{2}=\{3\}$, $b_{2}=4$, $\mathcal{A}_{3}=\{4\}$, $b_{3}=3$, i.e., we impose $V_{1}\perp\!\!\!\perp V_{2}$, $V_{2}\perp\!\!\!\perp V_{3}$, $V_{2}\stackrel{\iota}{\le}(V_{1},V_{3})$, $V_{4}\stackrel{\iota}{\le}V_{3}$, and $V_{3}\stackrel{\iota}{\le}V_{4}$. The red zig-zag edges are Network 1. The black solid edges are Network 2.

Theorems & Definitions (16)

• Theorem 1: verma1990causalgeiger1990logicgeiger1990d
• Theorem 2
• Corollary 3
• Definition 4: Sufficient statistic dawid1979conditional
• Lemma 5
• Lemma 6
• Lemma 7
• Theorem 8: li2023undecidability
• Lemma 9
• Theorem 10