Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ExDAG: an MIQP Algorithm for Learning DAGs

Pavel Rytir, Ales Wodecki, Jakub Marecek

TL;DR

This work tackles learning causal DAG structures by formulating the problem as a mixed-integer quadratic program (MIQP) that yields a maximum likelihood estimator under a structural equation model $X = XW + Z$ with $W$ acyclic. ExDAG solves the MIQP via a branch-and-bound-and-cut framework that adds cycle-exclusion constraints lazily, enabling global convergence without enumerating all cycles beforehand; dual bounds further allow real-time quality assessment. The method demonstrates superior structural Hamming distance and $F_1$ scores on medium-sized graphs (up to $d \le 25$) under Gaussian noise, and shows promising results on a NeurIPS real dataset, outperforming NOTEARS and other baselines. This approach offers scalable, provably global DAG learning with practical applicability in causal inference and related graphical-model tasks, and suggests avenues for future work on graph decompositions and pre-processing to further broaden scalability.

Abstract

There has been a growing interest in causal learning in recent years. Commonly used representations of causal structures, including Bayesian networks and structural equation models (SEM), take the form of directed acyclic graphs (DAGs). We provide a novel mixed-integer quadratic programming formulation and an associated algorithm that identifies DAGs with a low structural Hamming distance between the identified DAG and the ground truth, under identifiability assumptions. The eventual exact learning is guaranteed by the global convergence of the branch-and-bound-and-cut algorithm, which is utilized. In addition to this, integer programming techniques give us access to the dual bound, which allows for a real time assessment of the quality of solution. Previously, integer programming techniques have been shown to lead to limited scaling in the case of DAG identification due to the super exponential number of constraints, which prevent the formation of cycles. The algorithm proposed circumvents this by selectively generating only the violated constraints using the so-called "lazy" constraints methodology. Our empirical results show that ExDAG outperforms state-of-the-art solvers in terms of structural Hamming distance and $F_1$ score when considering Gaussian noise on medium-sized graphs.

ExDAG: an MIQP Algorithm for Learning DAGs

TL;DR

This work tackles learning causal DAG structures by formulating the problem as a mixed-integer quadratic program (MIQP) that yields a maximum likelihood estimator under a structural equation model with acyclic. ExDAG solves the MIQP via a branch-and-bound-and-cut framework that adds cycle-exclusion constraints lazily, enabling global convergence without enumerating all cycles beforehand; dual bounds further allow real-time quality assessment. The method demonstrates superior structural Hamming distance and scores on medium-sized graphs (up to ) under Gaussian noise, and shows promising results on a NeurIPS real dataset, outperforming NOTEARS and other baselines. This approach offers scalable, provably global DAG learning with practical applicability in causal inference and related graphical-model tasks, and suggests avenues for future work on graph decompositions and pre-processing to further broaden scalability.

Abstract

There has been a growing interest in causal learning in recent years. Commonly used representations of causal structures, including Bayesian networks and structural equation models (SEM), take the form of directed acyclic graphs (DAGs). We provide a novel mixed-integer quadratic programming formulation and an associated algorithm that identifies DAGs with a low structural Hamming distance between the identified DAG and the ground truth, under identifiability assumptions. The eventual exact learning is guaranteed by the global convergence of the branch-and-bound-and-cut algorithm, which is utilized. In addition to this, integer programming techniques give us access to the dual bound, which allows for a real time assessment of the quality of solution. Previously, integer programming techniques have been shown to lead to limited scaling in the case of DAG identification due to the super exponential number of constraints, which prevent the formation of cycles. The algorithm proposed circumvents this by selectively generating only the violated constraints using the so-called "lazy" constraints methodology. Our empirical results show that ExDAG outperforms state-of-the-art solvers in terms of structural Hamming distance and score when considering Gaussian noise on medium-sized graphs.
Paper Structure (12 sections, 14 equations, 3 figures)

This paper contains 12 sections, 14 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Related Work
  4. Problem Formulation and Identification Results
  5. Identification as a Mixed Integer Quadratic Problem
  6. The Branch-and-Bound-and-Cut Algorithm
  7. Numerical Results
  8. Setup Common to all the Benchmarking Experiments and Comparison Metrics
  9. Comparison of Identification Methods Under Gaussian Noise
  10. Finding the Optimal Running Time
  11. Identifying DAGs from Datasets with Real Interpretation using ExDAG
  12. Conclusion

Figures (3)

  • Figure 1: Structural Hamming distance for test cases using ER and SF random ensembles. Mean, standard deviation and minimum over 10 runs is depicted. Standard deviation is depicted as blurred area and the maximum as a dashed line.
  • Figure 2: $F_1$ score for test cases using ER and SF random ensembles. Mean, standard deviation and minimum over 10 runs is depicted. Standard deviation is depicted as blurred area and the minimum as a dashed line.
  • Figure 3: Convergence progress for the generation methods SF2, ER2, SF3, SF5