Neurosymbolic Diffusion Models

TL;DR

This work tackles the reliability and out-of-distribution robustness of neurosymbolic predictors by relaxing the typical conditional independence assumption between concepts. It introduces neurosymbolic diffusion models (NeSyDMs), which leverage masked discrete diffusion to capture dependencies among concepts while preserving local independence at each denoising step for scalable learning. The authors derive a continuous-time NeSyDM variational objective (the NeSyDM-NELBO) with three components—concept denoising, output denoising, and variational entropy—and develop gradient estimators and sampling schemes to train models that jointly reason over concepts and symbolic programs. Empirically, NeSyDMs achieve state-of-the-art accuracy among NeSy predictors on RSBench tasks and scale to high-dimensional reasoning challenges like visual path planning, while delivering improved calibration and RS-awareness, thus supporting safer and more reliable AI in complex domains.

Abstract

Neurosymbolic (NeSy) predictors combine neural perception with symbolic reasoning to solve tasks like visual reasoning. However, standard NeSy predictors assume conditional independence between the symbols they extract, thus limiting their ability to model interactions and uncertainty - often leading to overconfident predictions and poor out-of-distribution generalisation. To overcome the limitations of the independence assumption, we introduce neurosymbolic diffusion models (NeSyDMs), a new class of NeSy predictors that use discrete diffusion to model dependencies between symbols. Our approach reuses the independence assumption from NeSy predictors at each step of the diffusion process, enabling scalable learning while capturing symbol dependencies and uncertainty quantification. Across both synthetic and real-world benchmarks - including high-dimensional visual path planning and rule-based autonomous driving - NeSyDMs achieve state-of-the-art accuracy among NeSy predictors and demonstrate strong calibration.

Figures (2)

• Figure 1: NeSyDMs integrate masked diffusion models (orange boxes) with symbolic programs (blue box) to learn to predict the minimum cost path in a visual path-planning task. A variational posterior (\ref{['sec:variational_posterior']}) first obtains a candidate concept ${\mathbf{c}}^0$, that represents the costs of traversing each cell of the grid. Then, we partially mask ${\mathbf{c}}^0$ using the masking process $q({\mathbf{c}}^s\mid {\mathbf{c}}^0)$ to obtain masked concepts ${\mathbf{c}}^{\frac{1}{2}}$. We feed this to the discrete diffusion model's unmasking model$p_{\boldsymbol{\theta}}(\tilde{{\mathbf{c}}}\mid {\mathbf{x}}, {\mathbf{c}}^{\frac{1}{2}})$ to predict the unmasked concepts $\tilde{{\mathbf{c}}}^0$. We use the symbolic program $\varphi$, which we choose as Dijkstra's algorithm, to map the predicted concepts $\tilde{{\mathbf{c}}}^0$ to the predicted path $\tilde{{\mathbf{y}}}^0$. Finally, we use gradient estimation to update the parameters of the unmasking model. Dotted arrows denote samples from a distribution.
• Figure 2: Probabilistic graphical model for neurosymbolic diffusion model. The forward process $q$, indicated by striped arrows, masks both concepts ${\mathbf{c}}$ and outputs ${\mathbf{y}}$. Since only ${\mathbf{y}}^0$ is observed, a variational distribution $q_{\boldsymbol{\theta}}$ has to predict ${\mathbf{c}}^0$ from ${\mathbf{y}}^0$ and ${\mathbf{x}}$. The reverse process, with regular arrows, unmasks both concepts ${\mathbf{c}}$ and outputs ${\mathbf{y}}$, transforming concepts into outputs at every time step.

Theorems & Definitions (15)

• Example 2.1: pogancicDifferentiationBlackboxCombinatorial2020
• Example 2.2
• Theorem 3.1
• Theorem C.1
• proof
• Lemma C.2
• proof
• Lemma C.3
• proof
• Lemma C.4