Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unsupervised Pairwise Causal Discovery on Heterogeneous Data using Mutual Information Measures

Alexandre Trilla, Nenad Mijatovic

TL;DR

The paper tackles pairwise causal discovery in heterogeneous data under an unsupervised framework built on Mutual Information measures within a linear additive-noise model. It integrates unconditional independence tests (Pearson's $\chi^2$ and the Total Information Coefficient, TIC) into a RESIT-like orientation scheme, selecting the test type based on data characteristics to handle mixed variable types. A ChaLearn cause-effect pair benchmark is used to quantify performance, revealing a statistically significant yet modest average improvement (ACE ≈ $3.36\%$) when allowing flexible test selection, underscoring the value of unsupervised, MI-based methods in unknown environments. Overall, the work establishes a first standard baseline for two-variable causal discovery in heterogeneous settings while highlighting limitations due to data size and test coverage.

Abstract

A fundamental task in science is to determine the underlying causal relations because it is the knowledge of this functional structure what leads to the correct interpretation of an effect given the apparent associations in the observed data. In this sense, Causal Discovery is a technique that tackles this challenge by analyzing the statistical properties of the constituent variables. In this work, we target the generalizability of the discovery method by following a reductionist approach that only involves two variables, i.e., the pairwise or bi-variate setting. We question the current (possibly misleading) baseline results on the basis that they were obtained through supervised learning, which is arguably contrary to this genuinely exploratory endeavor. In consequence, we approach this problem in an unsupervised way, using robust Mutual Information measures, and observing the impact of the different variable types, which is oftentimes ignored in the design of solutions. Thus, we provide a novel set of standard unbiased results that can serve as a reference to guide future discovery tasks in completely unknown environments.

Unsupervised Pairwise Causal Discovery on Heterogeneous Data using Mutual Information Measures

TL;DR

The paper tackles pairwise causal discovery in heterogeneous data under an unsupervised framework built on Mutual Information measures within a linear additive-noise model. It integrates unconditional independence tests (Pearson's and the Total Information Coefficient, TIC) into a RESIT-like orientation scheme, selecting the test type based on data characteristics to handle mixed variable types. A ChaLearn cause-effect pair benchmark is used to quantify performance, revealing a statistically significant yet modest average improvement (ACE ≈ ) when allowing flexible test selection, underscoring the value of unsupervised, MI-based methods in unknown environments. Overall, the work establishes a first standard baseline for two-variable causal discovery in heterogeneous settings while highlighting limitations due to data size and test coverage.

Abstract

A fundamental task in science is to determine the underlying causal relations because it is the knowledge of this functional structure what leads to the correct interpretation of an effect given the apparent associations in the observed data. In this sense, Causal Discovery is a technique that tackles this challenge by analyzing the statistical properties of the constituent variables. In this work, we target the generalizability of the discovery method by following a reductionist approach that only involves two variables, i.e., the pairwise or bi-variate setting. We question the current (possibly misleading) baseline results on the basis that they were obtained through supervised learning, which is arguably contrary to this genuinely exploratory endeavor. In consequence, we approach this problem in an unsupervised way, using robust Mutual Information measures, and observing the impact of the different variable types, which is oftentimes ignored in the design of solutions. Thus, we provide a novel set of standard unbiased results that can serve as a reference to guide future discovery tasks in completely unknown environments.
Paper Structure (14 sections, 4 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 14 sections, 4 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Identifiability of a Linear Model with Non-Gaussian Noise
  4. Mutual Information as a Measure of Association
  5. Pearson's $\chi^2$ as an Unconditional Independence Test
  6. Total Information Coefficient as a Robust Independence Test
  7. Method
  8. Results
  9. Reference Benchmark Dataset
  10. Performance Scores
  11. Discussion
  12. Average Causal Effect of the Flexible Test Selection
  13. Limitations
  14. Conclusion

Figures (2)

  • Figure 1: Distribution of Mutual Information between the regression residual and the related hypothetical cause for the different causal structures. The low-value overlap (i.e., MI$<$0.025) between the Causal and the Independent structures illustrates the independence between the noise and the cause. The density of the real-valued MI indicator has been estimated with a smoothed Gaussian kernel.
  • Figure 2: A schematic illustrating the difference between the gridding strategies (shown in dotted lines) introduced by the independence tests. (Left) Uniform 3-by-3 grid as is used by the $\chi^2$ test. (Right) Optimum 2-by-3 grid $G$ as is used by the TIC test.