Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Higher-Order Causal Structure Learning with Additive Models

James Enouen, Yujia Zheng, Ignavier Ng, Yan Liu, Kun Zhang

TL;DR

This work tackles causal structure learning with higher-order interactions by extending the causal additive model (CAM) to directed hypergraphs (HDAGs). It develops three hypergraph generalizations (undirected, directed classical, and directed additive noise), establishes identifiability results (hyper Markov equivalence and multi-dependence tests), and introduces HCAM, a greedy algorithm that combines CAM with SIAN-based hyperedge discovery. Empirical results on synthetic ANMs demonstrate that modeling higher-order interactions can yield improvements in some settings (notably 2D), while revealing substantial challenges in 3D due to sample complexity. The framework provides a principled path to capturing complex causal mechanisms beyond pairwise interactions and points to future work on latent confounding and broader identifiability in hypergraph-based causal models.

Abstract

Causal structure learning has long been the central task of inferring causal insights from data. Despite the abundance of real-world processes exhibiting higher-order mechanisms, however, an explicit treatment of interactions in causal discovery has received little attention. In this work, we focus on extending the causal additive model (CAM) to additive models with higher-order interactions. This second level of modularity we introduce to the structure learning problem is most easily represented by a directed acyclic hypergraph which extends the DAG. We introduce the necessary definitions and theoretical tools to handle the novel structure we introduce and then provide identifiability results for the hyper DAG, extending the typical Markov equivalence classes. We next provide insights into why learning the more complex hypergraph structure may actually lead to better empirical results. In particular, more restrictive assumptions like CAM correspond to easier-to-learn hyper DAGs and better finite sample complexity. We finally develop an extension of the greedy CAM algorithm which can handle the more complex hyper DAG search space and demonstrate its empirical usefulness in synthetic experiments.

Higher-Order Causal Structure Learning with Additive Models

TL;DR

This work tackles causal structure learning with higher-order interactions by extending the causal additive model (CAM) to directed hypergraphs (HDAGs). It develops three hypergraph generalizations (undirected, directed classical, and directed additive noise), establishes identifiability results (hyper Markov equivalence and multi-dependence tests), and introduces HCAM, a greedy algorithm that combines CAM with SIAN-based hyperedge discovery. Empirical results on synthetic ANMs demonstrate that modeling higher-order interactions can yield improvements in some settings (notably 2D), while revealing substantial challenges in 3D due to sample complexity. The framework provides a principled path to capturing complex causal mechanisms beyond pairwise interactions and points to future work on latent confounding and broader identifiability in hypergraph-based causal models.

Abstract

Causal structure learning has long been the central task of inferring causal insights from data. Despite the abundance of real-world processes exhibiting higher-order mechanisms, however, an explicit treatment of interactions in causal discovery has received little attention. In this work, we focus on extending the causal additive model (CAM) to additive models with higher-order interactions. This second level of modularity we introduce to the structure learning problem is most easily represented by a directed acyclic hypergraph which extends the DAG. We introduce the necessary definitions and theoretical tools to handle the novel structure we introduce and then provide identifiability results for the hyper DAG, extending the typical Markov equivalence classes. We next provide insights into why learning the more complex hypergraph structure may actually lead to better empirical results. In particular, more restrictive assumptions like CAM correspond to easier-to-learn hyper DAGs and better finite sample complexity. We finally develop an extension of the greedy CAM algorithm which can handle the more complex hyper DAG search space and demonstrate its empirical usefulness in synthetic experiments.

Paper Structure

This paper contains 30 sections, 5 theorems, 45 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Hypergraph Methods
  3. Undirected Models
  4. Directed Classical Models
  5. Directed Additive Noise Models
  6. Structure Identifiability
  7. Undirected Models
  8. Directed Classical Models
  9. Directed Additive Noise Models
  10. Algorithm
  11. Step 1: Candidate Search
  12. Step 2: Greedy Selection
  13. Step 3: Final Pruning
  14. Experiments
  15. Results
  16. ...and 15 more sections

Key Result

Theorem 1

The HDAG is identifiable up to the hyper Markov Equivalence classes (HMECs), consisting of all HDAGs with the same "body" and the same (unshielded) "multi-colliders", paralleling the existing result identifying DAGs up to their skeleton and (unshielded) colliders.

Figures (2)

  • Figure 1: A depiction of the distinguishing power for hypergraphs corresponding to the same DAG. The CAM assumption implies moving from panel a to d. The ANM assumption implies moving from panel c to f. We propose the more HDAG which includes both CAM and ANM as special cases.
  • Figure 2: A gradual refinement of the DAGs within a MEC to a stronger refinement of HDAGs based on Theorem \ref{['thm:hyper_MECs']} to an HMEC. There are $d=4$ variables with a fully-connected DAG structure. The 24 DAGs correspond with the 4! orderings of the four variables. The green triangles represents third-degree hyperedges in the body of an HDAG. Lack of arrows indicate uncertainty about the orientation of the edges or hyperedges (between members of the same equivalence class).

Theorems & Definitions (27)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Theorem 1
  • Theorem 2
  • Corollary 1
  • ...and 17 more