Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sample Complexity of Nonparametric Closeness Testing for Continuous Distributions and Its Application to Causal Discovery with Hidden Confounding

Fateme Jamshidi, Sina Akbari, Negar Kiyavash

Abstract

We study the problem of closeness testing for continuous distributions and its implications for causal discovery. Specifically, we analyze the sample complexity of distinguishing whether two multidimensional continuous distributions are identical or differ by at least $ε$ in terms of Kullback-Leibler (KL) divergence under non-parametric assumptions. To this end, we propose an estimator of KL divergence which is based on the von Mises expansion. Our closeness test attains optimal parametric rates under smoothness assumptions. Equipped with this test, which serves as a building block of our causal discovery algorithm to identify the causal structure between two multidimensional random variables, we establish sample complexity guarantees for our causal discovery method. To the best of our knowledge, this work is the first work that provides sample complexity guarantees for distinguishing cause and effect in multidimensional non-linear models with non-Gaussian continuous variables in the presence of unobserved confounding.

Sample Complexity of Nonparametric Closeness Testing for Continuous Distributions and Its Application to Causal Discovery with Hidden Confounding

Abstract

We study the problem of closeness testing for continuous distributions and its implications for causal discovery. Specifically, we analyze the sample complexity of distinguishing whether two multidimensional continuous distributions are identical or differ by at least in terms of Kullback-Leibler (KL) divergence under non-parametric assumptions. To this end, we propose an estimator of KL divergence which is based on the von Mises expansion. Our closeness test attains optimal parametric rates under smoothness assumptions. Equipped with this test, which serves as a building block of our causal discovery algorithm to identify the causal structure between two multidimensional random variables, we establish sample complexity guarantees for our causal discovery method. To the best of our knowledge, this work is the first work that provides sample complexity guarantees for distinguishing cause and effect in multidimensional non-linear models with non-Gaussian continuous variables in the presence of unobserved confounding.

Paper Structure

This paper contains 18 sections, 14 theorems, 49 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Outline of the paper
  3. Related Work
  4. KL Divergence and Von Mises Estimators
  5. Sample complexity in closeness testing and causal discovery
  6. Background
  7. Causal preliminaries
  8. Exponential Concentration for KL Divergence Estimation
  9. Closeness Testing of Two Continuous Distributions
  10. Determining Causal Relationships
  11. Testing for A to B Using Observational and Interventional Data
  12. Error analysis
  13. Complete Causal Discovery Algorithm
  14. Testing for A to B Without Observational Samples
  15. Conclusion
  16. ...and 3 more sections

Key Result

Proposition 0

Assume that $p$ belongs to the Hölder class$\Sigma(\beta,L)$ on $\mathcal{X}$ for some $\beta,L>0$ and that $K_d$ satisfies Assumption ass:kernel. Let $h = h_n = \Theta(n^{-\frac{1}{2\beta + d}})$. Then, there exist $C_1, C_2, \varepsilon_0>0$ and $n_0 \geq 0$ such that for all $n^{-\frac{\beta}{2\b

Figures (1)

  • Figure 1: Three causal relationships between correlated variables $A$ and $B$.

Theorems & Definitions (21)

  • Definition 1: Hölder class
  • Proposition 0: Exponential concentration of $\| p -\what{p}_h\|_{\infty}$
  • Remark 1
  • Theorem 1: Exponential concentration of $\klv$ in Eq. \ref{['eq: vm']}
  • Remark 2
  • Remark 3
  • Remark 4
  • Corollary 2: Sample Complexity of Closeness Testing
  • Lemma 1
  • Lemma 2: levin1985one; see also Fact A.2 in goldreich2014multiple
  • ...and 11 more