Factored space models: Towards causality between levels of abstraction

TL;DR

The paper introduces factored space models (FSMs) to model causality across abstraction levels, including both probabilistic and deterministic relationships, addressing a key limitation of traditional causal graphs. It defines structural independence within FSMs and proves a generalized soundness and completeness theorem, extending $d$-separation to settings with deterministic dependencies. The authors show how to construct FSMs from Bayesian networks, argue that FSMs can be more expressive than DAG-based models, and establish a formal equivalence between FSM structural independence and probabilistic independence across all product-form distributions. These results provide a robust framework for reasoning about cross-level causality and abstraction in intelligent systems, with potential implications for neural networks, representation learning, and interpretable AI.

Abstract

Causality plays an important role in understanding intelligent behavior, and there is a wealth of literature on mathematical models for causality, most of which is focused on causal graphs. Causal graphs are a powerful tool for a wide range of applications, in particular when the relevant variables are known and at the same level of abstraction. However, the given variables can also be unstructured data, like pixels of an image. Meanwhile, the causal variables, such as the positions of objects in the image, can be arbitrary deterministic functions of the given variables. Moreover, the causal variables may form a hierarchy of abstractions, in which the macro-level variables are deterministic functions of the micro-level variables. Causal graphs are limited when it comes to modeling this kind of situation. In the presence of deterministic relationships there is generally no causal graph that satisfies both the Markov condition and the faithfulness condition. We introduce factored space models as an alternative to causal graphs which naturally represent both probabilistic and deterministic relationships at all levels of abstraction. Moreover, we introduce structural independence and establish that it is equivalent to statistical independence in every distribution that factorizes over the factored space. This theorem generalizes the classical soundness and completeness theorem for d-separation.

Figures (3)

• Figure 1: Summary of our results. In a causal graph, the standard criterion for independence is d-separation. However, d-separation is undefined for variables that are deterministic functions of other variables, such as $A\coloneq X+Y+Z$. To address this issue, we introduce factored space models (FSMs) as an alternative to causal graphs. FSMs are based on expressing the sample space as a Cartesian product, and can be visualized as a hyper-rectangle. FSMs allow us to define structural independence as an independence criterion which is also defined for deterministic functions of variables. Further, we establish a theorem that generalizes the soundness and completeness of d-separation koller2009probabilistic to structural independence.
• Figure 2: This graph violates the faithfulness condition: $Y$ and $Z$ are not d-separated by $\vec{X}$, despite $Y\mathrel{\perp\!\!\!\perp} Z\mid \vec{X}$.
• Figure 3: Factored space with three factors.

Theorems & Definitions (94)

• Definition 4.1: Derived Variable
• Definition 4.2: Factored Space
• Definition 4.3: Factorizing Distribution
• Definition 4.4: Factored Space Model
• Definition 4.5: Disintegration
• Definition 4.6: History, Generation
• Lemma 4.6: History is minimal generating set
• proof : Proof (sketch)
• Lemma 4.6: History of joint variable
• Lemma 4.6: History of a variable is the union of the histories of events