Bayesian Network Structure Discovery Using Large Language Models
Yinghuan Zhang, Yufei Zhang, Parisa Kordjamshidi, Zijun Cui
TL;DR
The paper tackles the problem of learning Bayesian network structures with limited observational data by placing large language models at the core of the discovery process. It introduces a two-phase framework: PromptBN, a data-free phase that induces a DAG from variable metadata via meta-prompting, and ReActBN, a data-aware phase that uses a ReAct-style Reason-and-Act loop to refine the graph with structure scores such as $BIC$ while enforcing acyclicity. The approach yields strong empirical gains over both traditional data-driven methods and prior LLM-based methods, particularly in low-data regimes, and demonstrates competitive generalization to newer datasets without retraining. The work shows that maintaining the LLM in the loop for both generation and refinement can achieve efficient, accurate BN structure discovery with practical impact in settings where data is scarce or expensive to obtain.
Abstract
Understanding probabilistic relationships among variables is crucial for analyzing complex systems. Traditional structure learning methods often require extensive observational data and incur high computational costs. Recent studies have explored using large language models (LLMs) for structure learning, but most treat LLMs as auxiliary tools for pre-processing or post-processing, leaving the core learning process data-driven. In this work, we propose a unified framework for Bayesian network structure discovery that places LLMs at the center, supporting both data-free and data-aware settings. In the data-free case, we introduce \textbf{PromptBN} to query LLMs with metadata and efficiently uncover valid probabilistic relationships. When observational data are available, we introduce \textbf{ReActBN}, which integrates the ReAct reasoning paradigm with structure scores such as the Bayesian Information Criterion (BIC) for iterative refinement. Unlike prior methods that offload refinement to external algorithms, our framework maintains the LLM actively in the loop throughout the discovery process. Experiments demonstrate that our method significantly outperforms both existing LLM-based approaches and traditional data-driven algorithms, particularly in the low- or no-data scenario. Code is publicly available at {\texttt{\textcolor{magenta}{https://github.com/sherryzyh/prompt2bn}}}.