Directed Acyclic Graph Structure Learning from Dynamic Graphs

TL;DR

This work addresses DAG structure learning for dynamic graphs by jointly modeling contemporaneous intra-slice and time-lagged inter-slice dependencies within a SVAR-based SEM. It introduces GraphNOTEARS, a score-based framework that minimizes a least-squares loss with $\ell_1$ penalties on $\mathbf{W}$ and $\mathbf{P}$ and enforces acyclicity through the smooth constraint $h(\mathbf{W}) = \mathrm{tr}(e^{\mathbf{W} \circ \mathbf{W}}) - d$, solved via an augmented Lagrangian and standard solvers, followed by edge thresholding. The method demonstrates superior edge recovery on synthetic data relative to baselines and validates practical utility on Yelp based real-world dynamic graphs, where learned edges align with domain knowledge. This approach enables interpretable causal-like structure learning in settings with temporal evolution and relational interactions, with potential extensions to nonlinear dynamics and tensor-valued data.

Abstract

Estimating the structure of directed acyclic graphs (DAGs) of features (variables) plays a vital role in revealing the latent data generation process and providing causal insights in various applications. Although there have been many studies on structure learning with various types of data, the structure learning on the dynamic graph has not been explored yet, and thus we study the learning problem of node feature generation mechanism on such ubiquitous dynamic graph data. In a dynamic graph, we propose to simultaneously estimate contemporaneous relationships and time-lagged interaction relationships between the node features. These two kinds of relationships form a DAG, which could effectively characterize the feature generation process in a concise way. To learn such a DAG, we cast the learning problem as a continuous score-based optimization problem, which consists of a differentiable score function to measure the validity of the learned DAGs and a smooth acyclicity constraint to ensure the acyclicity of the learned DAGs. These two components are translated into an unconstraint augmented Lagrangian objective which could be minimized by mature continuous optimization techniques. The resulting algorithm, named GraphNOTEARS, outperforms baselines on simulated data across a wide range of settings that may encounter in real-world applications. We also apply the proposed approach on two dynamic graphs constructed from the real-world Yelp dataset, demonstrating our method could learn the connections between node features, which conforms with the domain knowledge.

Figures (6)

• Figure 1: A toy example of feature generation process on a dynamic graph.
• Figure 2: Illustration of intra-slice (solid lines) and inter-slice (dashed lines) dependencies in a dynamic graph with $n=3$ samples and $d = 3$ variables at each timestamp and time-lagged order $p = 2$. For clarity, we ignore the edges that do not influence the variables $\mathbf{X}_{11}^{(t)}$.
• Figure 3: Example results for data with Gaussian noise, $n = 500$ samples, $d = 5$ variables at each timestamp, $T=7$ time-series, and $p = 2$ time-lagged graph effect order. Our algorithm recovers weights that are closer to the ground truth than baselines.
• Figure 4: F1 scores (higher is better) for different noise models (Gaussian, Exponential) and different sample sizes ($n\in[100, 200, 500]$). The length of time-series is 7 and we consider 1-step time-lagged neighbor influence here. Each panel contains results for two different choices of intra-slice graphs (columns) and inter-slice graphs (rows). Every marker corresponds to the mean performance across 5 algorithm runs, where each on a different simulated dataset, and shade area means the 95% confidence interval. Continuous and dashed lines represent F1 scores for intra-slice and inter-slice edges, respectively.
• Figure 5: SHD scores. Illustrations are the same as Fig. \ref{['fig:f1 results']}.

Theorems & Definitions (1)

• Definition 1: Dynamic Graph