Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Scalable and Valid Conditional Independence Testing with Spectral Representations

Alek Frohlich, Vladimir Kostic, Karim Lounici, Daniel Perazzo, Massimiliano Pontil

TL;DR

This work tackles scalable, nonparametric conditional independence testing by reframing CI through the partial covariance operator and learning its leading spectral features. A bi-level contrastive learning approach yields a rank-d spectral decomposition that feeds into an HSIC-like test statistic with an asymptotic chi-square null distribution, offering both validity and power guarantees. The framework adapts to diverse structural settings via representation learning, bridging kernel-based theory with modern neural methods and enabling GPU-accelerated scalability. Non-asymptotic power results depend on eigenvalue decay and representation error, with preliminary synthetic experiments indicating practical potential across high-dimensional conditioning scenarios.

Abstract

Conditional independence (CI) is central to causal inference, feature selection, and graphical modeling, yet it is untestable in many settings without additional assumptions. Existing CI tests often rely on restrictive structural conditions, limiting their validity on real-world data. Kernel methods using the partial covariance operator offer a more principled approach but suffer from limited adaptivity, slow convergence, and poor scalability. In this work, we explore whether representation learning can help address these limitations. Specifically, we focus on representations derived from the singular value decomposition of the partial covariance operator and use them to construct a simple test statistic, reminiscent of the Hilbert-Schmidt Independence Criterion (HSIC). We also introduce a practical bi-level contrastive algorithm to learn these representations. Our theory links representation learning error to test performance and establishes asymptotic validity and power guarantees. Preliminary experiments suggest that this approach offers a practical and statistically grounded path toward scalable CI testing, bridging kernel-based theory with modern representation learning.

Toward Scalable and Valid Conditional Independence Testing with Spectral Representations

TL;DR

This work tackles scalable, nonparametric conditional independence testing by reframing CI through the partial covariance operator and learning its leading spectral features. A bi-level contrastive learning approach yields a rank-d spectral decomposition that feeds into an HSIC-like test statistic with an asymptotic chi-square null distribution, offering both validity and power guarantees. The framework adapts to diverse structural settings via representation learning, bridging kernel-based theory with modern neural methods and enabling GPU-accelerated scalability. Non-asymptotic power results depend on eigenvalue decay and representation error, with preliminary synthetic experiments indicating practical potential across high-dimensional conditioning scenarios.

Abstract

Conditional independence (CI) is central to causal inference, feature selection, and graphical modeling, yet it is untestable in many settings without additional assumptions. Existing CI tests often rely on restrictive structural conditions, limiting their validity on real-world data. Kernel methods using the partial covariance operator offer a more principled approach but suffer from limited adaptivity, slow convergence, and poor scalability. In this work, we explore whether representation learning can help address these limitations. Specifically, we focus on representations derived from the singular value decomposition of the partial covariance operator and use them to construct a simple test statistic, reminiscent of the Hilbert-Schmidt Independence Criterion (HSIC). We also introduce a practical bi-level contrastive algorithm to learn these representations. Our theory links representation learning error to test performance and establishes asymptotic validity and power guarantees. Preliminary experiments suggest that this approach offers a practical and statistically grounded path toward scalable CI testing, bridging kernel-based theory with modern representation learning.
Paper Structure (26 sections, 10 theorems, 144 equations, 3 figures, 1 algorithm)

This paper contains 26 sections, 10 theorems, 144 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Learning Partial Covariance Operators
  4. CI Testing with Spectral Representations
  5. Conditional independence test (\ref{['fig:SCIT_diagram']}).
  6. Related Work
  7. Experiments
  8. Synthetic Data
  9. Conclusions
  10. Derivation of Losses and Algorithm for Learning the Partial Covariance Operator
  11. Proofs of Statistical Results
  12. Background on statistical hypothesis testing
  13. Statistical hypothesis testing.
  14. Background on sub-Gaussian random variables
  15. Background on Lindeberg-Feller Multivariate Central Limit Theorem
  16. ...and 11 more sections

Key Result

Theorem 4.1

Let Assumption ass:subgaussian be satisfied. Assume in addition that $\mathcal{E}_m \rightarrow 0$ as $m\rightarrow \infty$. Then under the null hypothesis, $\widehat{T}_n= n \, \lVert \widehat{C}_{\widehat{U}_\theta\widehat{V}_\theta} - \widehat{C}_{\widehat{U}_\theta\widehat{W}_\theta} \widehat{C}

Figures (3)

  • Figure 1: SCIT's testing pipeline. First the features $\widehat{u}_\theta,\widehat{v}_\theta,\widehat{w}_\theta$ are learned using \ref{['alg:learning_pcov']} with whitening over the training set. Then, the test statistic $\widehat{T}_n$ is computing using \ref{['eq:test_statistic']} over the test set. Finally, a decision is made using the $1-\alpha$ quantile of the chi-squared distribution with $d^2$ degrees of freedom, where $d$ is the output dimension of the networks.
  • Figure 2: Type 1 error.
  • Figure 3: Power.

Theorems & Definitions (16)

  • Theorem 4.1: Validity
  • Theorem 4.1: Validity
  • Theorem 4.2: Power
  • Theorem 1.1: Eckart-Young-Mirsky: loss form
  • Definition 2.1: Sub-Gaussian random vector
  • Lemma 2.1: (Sub-Gaussian random variable) Lemma 5.5. in vershynin2011introduction
  • Theorem 2.1: Multivariate Central Limit Theorem
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • ...and 6 more