Learning Identifiable Structures Helps Avoid Bias in DNN-based Supervised Causal Learning
Jiaru Zhang, Rui Ding, Qiang Fu, Bojun Huang, Zizhen Deng, Yang Hua, Haibing Guan, Shi Han, Dongmei Zhang
TL;DR
The paper addresses systematic bias in Deep Neural Network–based supervised causal learning (SCL) arising from predicting directed edges independently (the Node-Edge paradigm). It proposes SiCL, which predicts MEC-identifiable structures—a skeleton and a set of v-structures—via a pairwise-encoded architecture with unidirectional attention, enabling consistent estimators under the canonical MEC setting. The authors prove the identifiability advantage of skeleton and v-structures over edges, and demonstrate through extensive experiments on synthetic data and the Sachs benchmark that SiCL substantially outperforms state-of-the-art DNN-based SCL methods, including notable gains in SHD on real data. The work suggests a shift toward learning identifiable causal structures to mitigate fundamental bias and achieve robust causal discovery in practice, while acknowledging computational considerations and avenues for future improvements.
Abstract
Causal discovery is a structured prediction task that aims to predict causal relations among variables based on their data samples. Supervised Causal Learning (SCL) is an emerging paradigm in this field. Existing Deep Neural Network (DNN)-based methods commonly adopt the "Node-Edge approach", in which the model first computes an embedding vector for each variable-node, then uses these variable-wise representations to concurrently and independently predict for each directed causal-edge. In this paper, we first show that this architecture has some systematic bias that cannot be mitigated regardless of model size and data size. We then propose SiCL, a DNN-based SCL method that predicts a skeleton matrix together with a v-tensor (a third-order tensor representing the v-structures). According to the Markov Equivalence Class (MEC) theory, both the skeleton and the v-structures are identifiable causal structures under the canonical MEC setting, so predictions about skeleton and v-structures do not suffer from the identifiability limit in causal discovery, thus SiCL can avoid the systematic bias in Node-Edge architecture, and enable consistent estimators for causal discovery. Moreover, SiCL is also equipped with a specially designed pairwise encoder module with a unidirectional attention layer to model both internal and external relationships of pairs of nodes. Experimental results on both synthetic and real-world benchmarks show that SiCL significantly outperforms other DNN-based SCL approaches.