Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Fast Kernel-based Conditional Independence test with Application to Causal Discovery

Oliver Schacht, Biwei Huang

TL;DR

This work tackles the cubic-time barrier of kernel-based conditional independence testing (KCI) in causal discovery by introducing FastKCI, a scalable, parallelizable method that partitions the conditioning set Z via a mixture-of-experts Gaussian mixture model, runs local KCI tests within partitions, and aggregates results with importance weights to recover the global statistic. FastKCI preserves the KCI null distribution and statistical power while delivering substantial runtime improvements, particularly on large-scale data. Empirical results show comparable Type I error control and power to KCI across synthetic and production datasets, along with dramatic speedups and improved scalability over standard KCI and RCIT. The approach enables practical causal discovery on big data, though it relies on MoE-based partitioning and warrants further exploration for large conditioning sets and more robust partitioning schemes.

Abstract

Kernel-based conditional independence (KCI) testing is a powerful nonparametric method commonly employed in causal discovery tasks. Despite its flexibility and statistical reliability, cubic computational complexity limits its application to large datasets. To address this computational bottleneck, we propose \textit{FastKCI}, a scalable and parallelizable kernel-based conditional independence test that utilizes a mixture-of-experts approach inspired by embarrassingly parallel inference techniques for Gaussian processes. By partitioning the dataset based on a Gaussian mixture model over the conditioning variables, FastKCI conducts local KCI tests in parallel, aggregating the results using an importance-weighted sampling scheme. Experiments on synthetic datasets and benchmarks on real-world production data validate that FastKCI maintains the statistical power of the original KCI test while achieving substantial computational speedups. FastKCI thus represents a practical and efficient solution for conditional independence testing in causal inference on large-scale data.

A Fast Kernel-based Conditional Independence test with Application to Causal Discovery

TL;DR

This work tackles the cubic-time barrier of kernel-based conditional independence testing (KCI) in causal discovery by introducing FastKCI, a scalable, parallelizable method that partitions the conditioning set Z via a mixture-of-experts Gaussian mixture model, runs local KCI tests within partitions, and aggregates results with importance weights to recover the global statistic. FastKCI preserves the KCI null distribution and statistical power while delivering substantial runtime improvements, particularly on large-scale data. Empirical results show comparable Type I error control and power to KCI across synthetic and production datasets, along with dramatic speedups and improved scalability over standard KCI and RCIT. The approach enables practical causal discovery on big data, though it relies on MoE-based partitioning and warrants further exploration for large conditioning sets and more robust partitioning schemes.

Abstract

Kernel-based conditional independence (KCI) testing is a powerful nonparametric method commonly employed in causal discovery tasks. Despite its flexibility and statistical reliability, cubic computational complexity limits its application to large datasets. To address this computational bottleneck, we propose \textit{FastKCI}, a scalable and parallelizable kernel-based conditional independence test that utilizes a mixture-of-experts approach inspired by embarrassingly parallel inference techniques for Gaussian processes. By partitioning the dataset based on a Gaussian mixture model over the conditioning variables, FastKCI conducts local KCI tests in parallel, aggregating the results using an importance-weighted sampling scheme. Experiments on synthetic datasets and benchmarks on real-world production data validate that FastKCI maintains the statistical power of the original KCI test while achieving substantial computational speedups. FastKCI thus represents a practical and efficient solution for conditional independence testing in causal inference on large-scale data.
Paper Structure (34 sections, 1 theorem, 15 equations, 5 figures, 12 tables, 2 algorithms)

This paper contains 34 sections, 1 theorem, 15 equations, 5 figures, 12 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Covariance and conditional covariance operators
  5. Finite-sample KCI statistic
  6. Computational Complexity of the KCI
  7. FastKCI: A Scalable and Parallel Kernel-based Conditional Independence Test
  8. Partition Sampling.
  9. Local RKHS Embedding, Test Statistic and Null Distribution.
  10. Aggregation over $V$.
  11. Aggregation over $J$.
  12. Theoretical Insight
  13. Computational Complexity
  14. Hyperparameter Selection
  15. Experiments
  16. ...and 19 more sections

Key Result

Proposition 1

With characteristic kernels, $X\perp\!\!\!\!\perp Y\mid Z \Longleftrightarrow \Sigma_{XY\mid Z}=0 .$

Figures (5)

  • Figure 1: Motivation of the partitioning scheme in the data. The full covariance kernel estimation (left) is inefficient, while partitioning the data into components a single time (middle) may neglect some of the covariance structure. We propose to use multiple partitions $J$ (right) in parallel. We combine them using importance sampling. The figure is inspired by zhang2020presentation.
  • Figure 2: Simulated Type-I-Error ("Coverage") of the FastKCI and the KCI at 1% and 5% levels.
  • Figure 3: Power of FastKCI in different configurations compared to KCI. The violation of the null-hypothesis is increasing on the x-axis.
  • Figure 4: Precision, recall and F1-Score for KCI and FastKCI in causal discovery with growing sample size in setting A (left) and setting B (right).
  • Figure 5: Computation time with increasing sample size for (a) conditional independence testing and (b) causal discovery with the PC algorithm.

Theorems & Definitions (1)

  • Proposition 1: Fukumizu et al., 2007