Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Knowledge-Informed Pretrained Model for Causal Discovery

Wenbo Xu, Yue He, Yunhai Wang, Xingxuan Zhang, Kun Kuang, Yueguo Chen, Peng Cui

Abstract

Causal discovery has been widely studied, yet many existing methods rely on strong assumptions or fall into two extremes: either depending on costly interventional signals or partial ground truth as strong priors, or adopting purely data driven paradigms with limited guidance, which hinders practical deployment. Motivated by real-world scenarios where only coarse domain knowledge is available, we propose a knowledge-informed pretrained model for causal discovery that integrates weak prior knowledge as a principled middle ground. Our model adopts a dual source encoder-decoder architecture to process observational data in a knowledge-informed way. We design a diverse pretraining dataset and a curriculum learning strategy that smoothly adapts the model to varying prior strengths across mechanisms, graph densities, and variable scales. Extensive experiments on in-distribution, out-of distribution, and real-world datasets demonstrate consistent improvements over existing baselines, with strong robustness and practical applicability.

A Knowledge-Informed Pretrained Model for Causal Discovery

Abstract

Causal discovery has been widely studied, yet many existing methods rely on strong assumptions or fall into two extremes: either depending on costly interventional signals or partial ground truth as strong priors, or adopting purely data driven paradigms with limited guidance, which hinders practical deployment. Motivated by real-world scenarios where only coarse domain knowledge is available, we propose a knowledge-informed pretrained model for causal discovery that integrates weak prior knowledge as a principled middle ground. Our model adopts a dual source encoder-decoder architecture to process observational data in a knowledge-informed way. We design a diverse pretraining dataset and a curriculum learning strategy that smoothly adapts the model to varying prior strengths across mechanisms, graph densities, and variable scales. Extensive experiments on in-distribution, out-of distribution, and real-world datasets demonstrate consistent improvements over existing baselines, with strong robustness and practical applicability.
Paper Structure (30 sections, 6 equations, 6 figures, 4 tables)

This paper contains 30 sections, 6 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary Study
  4. Problem of Causal Discovery
  5. Motivation
  6. Knowledge Encoding
  7. Model Architecture
  8. Dual-Source Encoder
  9. Dual-Source Alignment
  10. Alternating Attention Mechanism
  11. Cross Attention Mechanism
  12. Graph Decoder
  13. Permutation Matrices
  14. Lower Triangle Decoder
  15. Model Training
  16. ...and 15 more sections

Figures (6)

  • Figure 1: Simple causality in chemistry. (A) Expert knowledge is structured as graph. (B) The corresponding ground-truth causal graph.
  • Figure 2: An illustration of knowledge encoding. (A) A symbolic form of Fig. \ref{['fig:examples_industry']}. (B) The corresponding adjacency matrix $\mathbf{A}$. (C) The reachability matrix $\mathbf{R}$, which further encodes indirect relations (e.g., $R_{A,C}=1$). (D) The resulting prior knowledge $G_P$, where infeasible relations are excluded and self-loops are removed.
  • Figure 3: (A) The overall pipeline of Kode, illustrating the global framework that integrates raw data and prior knowledge to infer the causal graph; (B) The dual-source encoder, which achieves alignment and feature compression from hybrid inputs; (C) The graph decoder, responsible for mapping the latent representations back into the final causal structure.
  • Figure 4: Results on 20 nodes, covering i.d. linear cases and both i.d. & o.o.d nonlinear mechanisms.
  • Figure 5: Performance on Gaussian Process ER(3) across scales: Kode maintains a clear advantage; GES ($4^{th}$ bar) collapses owing to excessive density at $N=30$ and $40$, so its results are omitted.
  • ...and 1 more figures