Causal Discovery by Kernel Deviance Measures with Heterogeneous Transforms

TL;DR

This work tackles the problem of discovering the causal direction between two random variables by enhancing the ability to detect higher-order structure in conditional distributions. It introduces Kernel Intrinsic Invariance Measure with Heterogeneous Transformation (KIIM-HT), which uses instance-specific neural projections to map kernel mean embeddings into a subspace that preserves higher-order deviances, enabling a robust direction test via comparing empirical scores $\hat{\mathcal{S}}_{\bm{x} \rightarrow \bm{y}}$ and $\hat{\mathcal{S}}_{\bm{y} \rightarrow \bm{x}}$. A re-weighted variant (Rw-KIIM-HT) is proposed to handle noise and outliers, and the method is optimized with stochastic gradient descent, including a regularization term to avoid degenerate solutions. Empirical results on synthetic data, two-dimensional synthetic data, and the Tübingen Cause-Effect Pairs show KIIM-HT achieving high accuracy and low variance, often outperforming baselines, and demonstrating robustness to regularization and hyperparameter choices.

Abstract

The discovery of causal relationships in a set of random variables is a fundamental objective of science and has also recently been argued as being an essential component towards real machine intelligence. One class of causal discovery techniques are founded based on the argument that there are inherent structural asymmetries between the causal and anti-causal direction which could be leveraged in determining the direction of causation. To go about capturing these discrepancies between cause and effect remains to be a challenge and many current state-of-the-art algorithms propose to compare the norms of the kernel mean embeddings of the conditional distributions. In this work, we argue that such approaches based on RKHS embeddings are insufficient in capturing principal markers of cause-effect asymmetry involving higher-order structural variabilities of the conditional distributions. We propose Kernel Intrinsic Invariance Measure with Heterogeneous Transform (KIIM-HT) which introduces a novel score measure based on heterogeneous transformation of RKHS embeddings to extract relevant higher-order moments of the conditional densities for causal discovery. Inference is made via comparing the score of each hypothetical cause-effect direction. Tests and comparisons on a synthetic dataset, a two-dimensional synthetic dataset and the real-world benchmark dataset Tübingen Cause-Effect Pairs verify our approach. In addition, we conduct a sensitivity analysis to the regularization parameter to faithfully compare previous work to our method and an experiment with trials on varied hyperparameter values to showcase the robustness of our algorithm.

Figures (1)

• Figure 1: Conditional distributions of the causal direction, $p(y|x)$, and the anti-causal direction, $p(x|y)$.