Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Covariate-dependent Graphical Model Estimation via Neural Networks with Statistical Guarantees

Jiahe Lin, Yikai Zhang, George Michailidis

TL;DR

This paper considers settings where the graph structure is covariate-dependent, and investigates a deep neural network-based approach to estimate it, which allows for flexible functional dependency on the covariate, and fits the data reasonably well in the absence of a Gaussianity assumption.

Abstract

Graphical models are widely used in diverse application domains to model the conditional dependencies amongst a collection of random variables. In this paper, we consider settings where the graph structure is covariate-dependent, and investigate a deep neural network-based approach to estimate it. The method allows for flexible functional dependency on the covariate, and fits the data reasonably well in the absence of a Gaussianity assumption. Theoretical results with PAC guarantees are established for the method, under assumptions commonly used in an Empirical Risk Minimization framework. The performance of the proposed method is evaluated on several synthetic data settings and benchmarked against existing approaches. The method is further illustrated on real datasets involving data from neuroscience and finance, respectively, and produces interpretable results.

Covariate-dependent Graphical Model Estimation via Neural Networks with Statistical Guarantees

TL;DR

This paper considers settings where the graph structure is covariate-dependent, and investigates a deep neural network-based approach to estimate it, which allows for flexible functional dependency on the covariate, and fits the data reasonably well in the absence of a Gaussianity assumption.

Abstract

Graphical models are widely used in diverse application domains to model the conditional dependencies amongst a collection of random variables. In this paper, we consider settings where the graph structure is covariate-dependent, and investigate a deep neural network-based approach to estimate it. The method allows for flexible functional dependency on the covariate, and fits the data reasonably well in the absence of a Gaussianity assumption. Theoretical results with PAC guarantees are established for the method, under assumptions commonly used in an Empirical Risk Minimization framework. The performance of the proposed method is evaluated on several synthetic data settings and benchmarked against existing approaches. The method is further illustrated on real datasets involving data from neuroscience and finance, respectively, and produces interpretable results.

Paper Structure

This paper contains 40 sections, 6 theorems, 75 equations, 6 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Non-Gaussian Graphical models.
  4. Covariate-dependent graphical models.
  5. Our Contributions
  6. DNN-based Covariate-Dependent Graphical Model Estimation
  7. Theoretical Results
  8. Preliminaries and A Road Map
  9. Definitions.
  10. Assumptions.
  11. Main Results
  12. Extensions to edge-wise recovery.
  13. Synthetic Data Experiments
  14. Settings.
  15. Performance evaluation.
  16. ...and 25 more sections

Key Result

Theorem 1

Under Assumptions assume2-assume3.5, the following bound for the excess error as a function of the empirical risk minimizer $\widehat{\boldsymbol{\beta}}$ (see ermdef) based on $S_n$ samples holds with probability at least $1-\delta$, for any $\delta>0$:

Figures (6)

  • Figure 1: Estimated networks as represented by $-\widehat{\beta}(\boldsymbol{z})$ for subjects having low (left) and high scores (right), after thresholding at 0.05. Red cells indicate positive partial correlations and blue cells indicate negative ones.
  • Figure 2: Pictorial illustration for settings G1,G2,N1,N2. Left: candidate skeletons $\Psi_1,\Psi_2,\Psi_3$. Right: different $\Theta^i$'s that are obtained from candidate skeletons, with the exact mixing depending on the values of the corresponding $\boldsymbol{z}^i$'s; their diagonals are suppressed for visualization purpose.
  • Figure 3: Pictorial illustration for setting D1. Left: candidate skeletons $B_1, B_2$ and an example of their convex combination. Right: Moralized graphs for the respective DAGs, with their diagonals suppressed for visualization purpose.
  • Figure 4: Left panel: various connectivity metrics of the estimated graphs (top) and the corresponding VIX level (bottom), from the beginning of 2001 to the end of 2023. Notably events that significantly increased market volatility have been marked. Right panel: heatmaps of the estimated graphs on two representative dates that respectively have high and low-VIX, with red cells indicating positive partial correlation and blue cells negative ones.
  • Figure 5: Averaged partial correlation graphs for high-VIX (left) and normal-VIX (right) days, with red cells indicating positive partial correlation and blue cells negative ones.
  • ...and 1 more figures

Theorems & Definitions (26)

  • Remark 1: On the implementation of $\beta_{jk}(\cdot)$ networks
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Theorem 1
  • Remark 6
  • Corollary 1: Consistency of $\widehat{\boldsymbol{\beta}}$
  • Theorem 2
  • Remark 7
  • ...and 16 more