Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heckerthoughts

David Heckerman

TL;DR

The paper provides a personal yet technical overview of foundational ideas in uncertainty, decision making, and graphical models, tracing the historical shift from non-probabilistic AI to probabilistic reasoning and Bayesian networks. It presents methods for learning acausal and causal graphical models from observational and interventional data, discusses intervention through do-operators and instrumental variables, and articulates strategies to handle confounders. The work showcases practical applications at Microsoft Research, from expert systems and spam filters to collaborative filtering, as well as healthcare–oriented genomics and vaccine design, illustrating the real-world impact of probabilistic graphical modeling. Overall, it argues that probabilistic reasoning, causal structure, and modular inference underpin robust decision making in AI, providing both methodological guidance and narrative context for the early adoption of these ideas. The synthesis emphasizes the importance of probability in enabling reliable uncertainty management and causal inference across diverse domains.

Abstract

This manuscript is technical memoir about my work at Stanford and Microsoft Research. Included are fundamental concepts central to machine learning and artificial intelligence, applications of these concepts, and stories behind their creation.

Heckerthoughts

TL;DR

The paper provides a personal yet technical overview of foundational ideas in uncertainty, decision making, and graphical models, tracing the historical shift from non-probabilistic AI to probabilistic reasoning and Bayesian networks. It presents methods for learning acausal and causal graphical models from observational and interventional data, discusses intervention through do-operators and instrumental variables, and articulates strategies to handle confounders. The work showcases practical applications at Microsoft Research, from expert systems and spam filters to collaborative filtering, as well as healthcare–oriented genomics and vaccine design, illustrating the real-world impact of probabilistic graphical modeling. Overall, it argues that probabilistic reasoning, causal structure, and modular inference underpin robust decision making in AI, providing both methodological guidance and narrative context for the early adoption of these ideas. The synthesis emphasizes the importance of probability in enabling reliable uncertainty management and causal inference across diverse domains.

Abstract

This manuscript is technical memoir about my work at Stanford and Microsoft Research. Included are fundamental concepts central to machine learning and artificial intelligence, applications of these concepts, and stories behind their creation.
Paper Structure (20 sections, 39 equations, 37 figures, 1 table)

This paper contains 20 sections, 39 equations, 37 figures, 1 table.

Table of Contents

  1. Introduction
  2. Uncertainty and AI's transition to probability
  3. The inevitability of probability
  4. Independence and graphical models
  5. AI’s transition to probability
  6. Decision making and causal reasoning
  7. Decision theory, influence diagrams, and decision trees
  8. Proper scoring rules
  9. Correlation versus causation and a definition of causality
  10. Fully causal models and the force/set/do decision
  11. The modularity of cause--effect relationships
  12. Learning graphical models from data
  13. Learning an acausal DAG model for one finite variable
  14. Learning acausal DAG models for two or more finite variables
  15. Learning fully causal models from observational data only
  16. ...and 5 more sections

Figures (37)

  • Figure 1: A photo of my visit with I.J. Good in 1987. From left to right: Greg Cooper, Eric Horvitz, I.J. Good, and me.
  • Figure 2: The wheel of fortune: a tool for assessing degrees of belief.
  • Figure 3: A thumbtack having landed heads.
  • Figure 4: A Bayesian network for detecting credit-card fraud. Arcs are drawn from cause to effect. The local probability distribution(s) associated with a node are shown adjacent to the node. Asterisks are wild cards---a shorthand for any value.
  • Figure 5: A simple Certainty Factor model for diagnosing infection.
  • ...and 32 more figures