Competition is the key: A Game Theoretic Causal Discovery Approach

TL;DR

This work tackles causal discovery from observational data by bridging scalability and finite-sample guarantees through a game-theoretic reinforcement learning framework. The proposed DDQN-CD agent refines a warm-start graph supplied by a strong opponent (GES or GraN-DAG) via acyclicity-constrained edge edits, guided by a Copula-BIC/BIC score and trained with Double DQN in a champion–challenger setup. The authors prove three guarantees: safety (never worse than the opponent), accelerated convergence from warm-starts, and high-probability recovery of the true best candidate as data grows, with exponential decay in mis-selection probability demonstrated on synthetic data and strong scalability on real benchmarks up to graphs with hundreds of nodes. Empirically, DDQN-CD matches or exceeds baselines across small, mid-sized, and large networks (Asia, Sachs, Alarm, Child, Hepar2, Dream, Andes), while maintaining theoretical safety and consistency. The work thus presents a practical, scalable, and theoretically grounded RL-based approach that advances the unification of empirical performance and finite-sample theory in causal discovery.

Abstract

Causal discovery remains a central challenge in machine learning, yet existing methods face a fundamental gap: algorithms like GES and GraN-DAG achieve strong empirical performance but lack finite-sample guarantees, while theoretically principled approaches fail to scale. We close this gap by introducing a game-theoretic reinforcement learning framework for causal discovery, where a DDQN agent directly competes against a strong baseline (GES or GraN-DAG), always warm-starting from the opponent's solution. This design yields three provable guarantees: the learned graph is never worse than the opponent, warm-starting strictly accelerates convergence, and most importantly, with high probability the algorithm selects the true best candidate graph. To the best of our knowledge, our result makes a first-of-its-kind progress in explaining such finite-sample guarantees in causal discovery: on synthetic SEMs (30 nodes), the observed error probability decays with n, tightly matching theory. On real-world benchmarks including Sachs, Asia, Alarm, Child, Hepar2, Dream, and Andes, our method consistently improves upon GES and GraN-DAG while remaining theoretically safe. Remarkably, it scales to large graphs such as Hepar2 (70 nodes), Dream (100 nodes), and Andes (220 nodes). Together, these results establish a new class of RL-based causal discovery algorithms that are simultaneously provably consistent, sample-efficient, and practically scalable, marking a decisive step toward unifying empirical performance with rigorous finite-sample theory.

Figures (4)

• Figure 1: Summary of our contributions. DDQN-CD integrates a unified RL-based framework with theoretical guarantees and demonstrates scalability across diverse benchmarks.
• Figure 2: DDQN--CD framework. Observational data and a warm start are merged and fed to a DDQN agent that proposes edge edits; an environment evaluates candidates via BIC/Copula-BIC and returns a reward until the stopping condition triggers, yielding the discovered DAG.
• Figure 3: Synthetic verification of Theorem \ref{['thm:champion']}. The mis-selection probability (blue, left axis) decays with $n$, while the gap $\Delta_n$ (green, right axis) grows with $n$, matching theoretical predictions.
• Figure 4: Composite Score vs. dataset size across all algorithms (Asia$\rightarrow$Andes). ★ marks datasets where Ours attains the best score (ties included).

Theorems & Definitions (17)

• Theorem 1: Never worse than opponent
• proof
• Definition 1: 1-optimal DAG
• Theorem 2: Hitting time bound
• proof
• Theorem 3: High-probability champion selection
• proof
• Theorem 3: Never worse than opponent
• proof
• Lemma 1: Strictly improving path exists