Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Learning Should Embrace the Wisdom of the Crowd

Ryan Feng Lin, Yuantao Wei, Huiling Liao, Xiaoning Qian, Shuai Huang

TL;DR

This paper argues that causal learning is now ready for the emergence of a new paradigm supported by rapidly advancing technologies, fulfilling the long-standing vision of leveraging human causal knowledge.

Abstract

Learning causal structures typically represented by directed acyclic graphs (DAGs) from observational data is notoriously challenging due to the combinatorial explosion of possible graphs and inherent ambiguities in observations. This paper argues that causal learning is now ready for the emergence of a new paradigm supported by rapidly advancing technologies, fulfilling the long-standing vision of leveraging human causal knowledge. This paradigm integrates scalable crowdsourcing platforms for data collection, interactive knowledge elicitation for expert opinion modeling, robust aggregation techniques for expert reconciliation, and large language model (LLM)-based simulation for augmenting AI-driven information acquisition. In this paper, we focus on DAG learning for causal discovery and frame the problem as a distributed decision-making task, recognizing that each participant (human expert or LLM agent) possesses fragmented and imperfect knowledge about different subsets of the variables of interest in the causal graph. By proposing a systematic framework to synthesize these insights, we aim to enable the recovery of a global causal structure unachievable by any individual agent alone. We advocate for a new research frontier and outline a comprehensive framework for new research thrusts that range from eliciting, modeling, aggregating, and optimizing human causal knowledge contributions.

Causal Learning Should Embrace the Wisdom of the Crowd

TL;DR

This paper argues that causal learning is now ready for the emergence of a new paradigm supported by rapidly advancing technologies, fulfilling the long-standing vision of leveraging human causal knowledge.

Abstract

Learning causal structures typically represented by directed acyclic graphs (DAGs) from observational data is notoriously challenging due to the combinatorial explosion of possible graphs and inherent ambiguities in observations. This paper argues that causal learning is now ready for the emergence of a new paradigm supported by rapidly advancing technologies, fulfilling the long-standing vision of leveraging human causal knowledge. This paradigm integrates scalable crowdsourcing platforms for data collection, interactive knowledge elicitation for expert opinion modeling, robust aggregation techniques for expert reconciliation, and large language model (LLM)-based simulation for augmenting AI-driven information acquisition. In this paper, we focus on DAG learning for causal discovery and frame the problem as a distributed decision-making task, recognizing that each participant (human expert or LLM agent) possesses fragmented and imperfect knowledge about different subsets of the variables of interest in the causal graph. By proposing a systematic framework to synthesize these insights, we aim to enable the recovery of a global causal structure unachievable by any individual agent alone. We advocate for a new research frontier and outline a comprehensive framework for new research thrusts that range from eliciting, modeling, aggregating, and optimizing human causal knowledge contributions.
Paper Structure (29 sections, 8 equations, 4 figures, 5 tables)

This paper contains 29 sections, 8 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. History of Causal Knowledge Elicitation.
  4. Experimental Insights into Human Causal Knowledge: Heterogeneity and Reliability
  5. A Thought Experiment
  6. A Real-World Case Study
  7. Insights of the Real-World Study
  8. Characterizations of Knowledge Quality.
  9. Taxonomy of Expert Types.
  10. We Need Human Causal Knowledge Models
  11. Expert Causal Knowledge Models
  12. Edge-wise causal knowledge.
  13. Ordering-wise causal knowledge.
  14. Comparative Analysis: Edge vs. Ordering
  15. Extension of Elicitation Frameworks
  16. ...and 14 more sections

Figures (4)

  • Figure 1: Bayesian network Asia with eight nodes and eight causal links representing a simplified respiratory diagnosis scenario.
  • Figure 2: Expert-level (left) and query-level (right) causal knowledge complexity (sorted by median of absolute scores).
  • Figure 3: A toy example illustrating the different expert types. Solid edges represent the causal links asserted by the expert, including correct (black), reverse (yellow) and spurious (red) edges, while dashed edges indicate low-confidence causal links regardless of validity.
  • Figure 4: Illustration of expert types in four dimensions.

Theorems & Definitions (5)

  • Definition 1: Path
  • Definition 2: Directed Acyclic Graph (DAG)
  • Definition 3: Parent variable
  • Definition 4: Bayesian Network
  • Definition 5: Topological order