Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Relational Neurosymbolic Markov Models

Lennert De Smet, Gabriele Venturato, Luc De Raedt, Giuseppe Marra

TL;DR

This work introduces Relational Neurosymbolic Markov Models (NeSy-MMs), a framework that fuses relational symbolic reasoning with end-to-end differentiable sequential models to enforce logical constraints in dynamic settings. A differentiable Rao-Blackwellised particle filter and cluster-based inference enable scalable training and inference across hybrid (finite/infinite) domains, while supporting both discriminative and generative tasks. Empirical results on generative and discriminative NetHack/MiniHack-inspired benchmarks demonstrate improved logical consistency, interpretability, and out-of-distribution generalisation, with scalable performance on longer horizons compared to purely neural baselines. The approach offers a principled path to trustworthy, constraint-aware sequential AI with potential applications in safety-critical domains and reinforcement learning, where relational knowledge and test-time adaptation are valuable.

Abstract

Sequential problems are ubiquitous in AI, such as in reinforcement learning or natural language processing. State-of-the-art deep sequential models, like transformers, excel in these settings but fail to guarantee the satisfaction of constraints necessary for trustworthy deployment. In contrast, neurosymbolic AI (NeSy) provides a sound formalism to enforce constraints in deep probabilistic models but scales exponentially on sequential problems. To overcome these limitations, we introduce relational neurosymbolic Markov models (NeSy-MMs), a new class of end-to-end differentiable sequential models that integrate and provably satisfy relational logical constraints. We propose a strategy for inference and learning that scales on sequential settings, and that combines approximate Bayesian inference, automated reasoning, and gradient estimation. Our experiments show that NeSy-MMs can solve problems beyond the current state-of-the-art in neurosymbolic AI and still provide strong guarantees with respect to desired properties. Moreover, we show that our models are more interpretable and that constraints can be adapted at test time to out-of-distribution scenarios.

Relational Neurosymbolic Markov Models

TL;DR

This work introduces Relational Neurosymbolic Markov Models (NeSy-MMs), a framework that fuses relational symbolic reasoning with end-to-end differentiable sequential models to enforce logical constraints in dynamic settings. A differentiable Rao-Blackwellised particle filter and cluster-based inference enable scalable training and inference across hybrid (finite/infinite) domains, while supporting both discriminative and generative tasks. Empirical results on generative and discriminative NetHack/MiniHack-inspired benchmarks demonstrate improved logical consistency, interpretability, and out-of-distribution generalisation, with scalable performance on longer horizons compared to purely neural baselines. The approach offers a principled path to trustworthy, constraint-aware sequential AI with potential applications in safety-critical domains and reinforcement learning, where relational knowledge and test-time adaptation are valuable.

Abstract

Sequential problems are ubiquitous in AI, such as in reinforcement learning or natural language processing. State-of-the-art deep sequential models, like transformers, excel in these settings but fail to guarantee the satisfaction of constraints necessary for trustworthy deployment. In contrast, neurosymbolic AI (NeSy) provides a sound formalism to enforce constraints in deep probabilistic models but scales exponentially on sequential problems. To overcome these limitations, we introduce relational neurosymbolic Markov models (NeSy-MMs), a new class of end-to-end differentiable sequential models that integrate and provably satisfy relational logical constraints. We propose a strategy for inference and learning that scales on sequential settings, and that combines approximate Bayesian inference, automated reasoning, and gradient estimation. Our experiments show that NeSy-MMs can solve problems beyond the current state-of-the-art in neurosymbolic AI and still provide strong guarantees with respect to desired properties. Moreover, we show that our models are more interpretable and that constraints can be adapted at test time to out-of-distribution scenarios.

Paper Structure

This paper contains 36 sections, 22 equations, 10 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Markov Models
  4. Probabilistic Neurosymbolic AI
  5. Relational Neurosymbolic Markov Models
  6. NeSy-MMs explicitly model symbols and their relations.
  7. NeSy-MMs factorise symbols over sequences.
  8. NeSy-MMs integrate neural and logical parametrisations.
  9. NeSy-MMs express discriminative and generative neurosymbolic models.
  10. Inference and Learning
  11. Differentiable NeSy-MM Particle Filtering
  12. NeSy inference via cluster factorisation
  13. Experiments
  14. Generative
  15. Discriminative
  16. ...and 21 more sections

Figures (10)

  • Figure 1: Probabilistic graphical model representations of the different systems considered in this work. Blue represents the states, green the observations.
  • Figure 2: DeepSeaProbLog de2023neural encoding of Example \ref{['ex:nesy-game']}. The first two lines are neural predicates that represent deep random variables modelling the normally distributed locations of the monster problog.py:ProbLogLexer -xM and the player problog.py:ProbLogLexer -xP. Each of the neural predicates has a named neural network that takes the image problog.py:ProbLogLexer -xIm as input and outputs the parameters of its random variable. The third line introduces a Bernoulli random variable problog.py:ProbLogLexer -xclumsy indicating that the monster will be clumsy with a $75\%$ chance. The final two lines express two rules of the game that determine when the monster problog.py:ProbLogLexer -xMproblog.py:ProbLogLexer -xhits the player problog.py:ProbLogLexer -xP and when the image problog.py:ProbLogLexer -xIm depicts a lost game, when the image depicts the monster hitting the player.
  • Figure 3: On the left, a logic programming description of the game Example \ref{['ex:markov-game']} in the discrete-continuous probabilistic NeSy language DeepSeaProbLog. On the right, the corresponding graphical model. We use plate notation to indicate a Markov transition. A rolled-out version is available in the appendix in Figure \ref{['fig:markov-game:rolled-out']}.
  • Figure 4: Example trajectories of length $4$ in a $5{\times}5$ grid for the generative (a) and discriminative (b) datasets, with the corresponding labels above the images. Note that for the discriminative task, the models do not take images as input but rather the symbolic state. The images are provided for visualization purposes.
  • Figure 5: Generated trajectory for actions: right, down, left, up, left, up, right, down.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Example 2.1
  • Example 3.1
  • Example A.1: Logic Program
  • Example A.2: DeepSeaProbLog program
  • Example A.3