CausalDynamics: A large-scale benchmark for structural discovery of dynamical causal models

TL;DR

The paper tackles the lack of robust benchmarks for causal discovery in nonlinear dynamical systems where interventions are impractical. It introduces CausalDynamics, a tiered, extensible data-generation framework that yields thousands of ground-truth structural dynamical causal models from both ordinary/stochastic differential equations and pseudo-real climate models. Key contributions include the largest benchmark of over 14k graphs with ground-truth causal graphs, a novel Growing Network with Redirection graph generator, an integrated evaluation workflow, and baseline results showing DL methods lag behind simpler approaches in many settings. The framework enables robust development of causal-discovery methods capable of handling high-dimensional, nonlinear, time-lagged, and confounded dynamics across diverse domains, with practical impact for scientific inference under real-world constraints.

Abstract

Causal discovery for dynamical systems poses a major challenge in fields where active interventions are infeasible. Most methods used to investigate these systems and their associated benchmarks are tailored to deterministic, low-dimensional and weakly nonlinear time-series data. To address these limitations, we present CausalDynamics, a large-scale benchmark and extensible data generation framework to advance the structural discovery of dynamical causal models. Our benchmark consists of true causal graphs derived from thousands of both linearly and nonlinearly coupled ordinary and stochastic differential equations as well as two idealized climate models. We perform a comprehensive evaluation of state-of-the-art causal discovery algorithms for graph reconstruction on systems with noisy, confounded, and lagged dynamics. CausalDynamics consists of a plug-and-play, build-your-own coupling workflow that enables the construction of a hierarchy of physical systems. We anticipate that our framework will facilitate the development of robust causal discovery algorithms that are broadly applicable across domains while addressing their unique challenges. We provide a user-friendly implementation and documentation on https://kausable.github.io/CausalDynamics.

Figures (13)

• Figure 1: Illustration of the tiered framework in CausalDynamics consisting of a plug-and-play, build-your-own coupling workflow that enables the construction of a hierarchy of physical systems with common causal challenges, such as unobserved confounders, time-lags, and noisy time series.
• Figure 2: Illustration of the Rössler Oscillator ODEs (left), the associated adjacency matrix $\mathcal{A}$ (center) and the corresponding causal graph (right).
• Figure 3: (a) Hierarchically coupled graphs are sample as scale-free DAGs, where root nodes can either be driven by period functions (root node 1) or dynamical systems (root node 2). Information is passed to the leaf nodes (leaf node 0) through MLPs. To correct for varsortability node values can optionally be standardized. To create a diverse set of challenges we introduce (b) unobserved confounders (green dotted lines), or (c) time-lag $\tau$ (red node). The type of nonlinearity can also be varied by prescribing or randomly assigning different edge-level activation functions. Assigned drivers are plotted by solid black edges. Note that each node is $\in \mathbb{R}^d$ and each edge is a function $f: \mathbb{R}^d \rightarrow \mathbb{R}^d$.
• Figure 4: CDF of AUROC (top) and AUPRC (bottom) for mean values reported in Table \ref{['tab:summary_result']} for different baselines across coupled system experiments. Models that perform better yield a low area under the curve for both AUROC and AUPRC.
• Figure 5: Example of baseline performance for coupled systems ($n=5$) for causal challenges and ENSO in the decoupled Atlantic setting (climate). Inference is performed using the best performing algorithm. Grey nodes and edges represent unobserved confounder, and dashed lines denote time-lagged relationships.