Neurosymbolic Decision Trees

TL;DR

The paper tackles learning both the symbolic structure and neural parameters in neurosymbolic AI by introducing neurosymbolic decision trees (NDTs) and the NeuID3 learning algorithm. It merges top-down decision-tree induction with neural probabilistic logic (DeepProbLog) to support symbolic and subsymbolic inputs and to leverage background knowledge. Empirical results across UCI datasets, MNIST/EMNIST-based subsymbolic tasks, and Eleusis demonstrate that NLDTs can outperform purely neural approaches, particularly when background knowledge is provided and neural tests are reused. This work advances data-efficient, modular neurosymbolic models with interpretable structure, offering a natural path toward continual learning and knowledge-driven AI systems.

Abstract

Neurosymbolic (NeSy) AI studies the integration of neural networks (NNs) and symbolic reasoning based on logic. Usually, NeSy techniques focus on learning the neural, probabilistic and/or fuzzy parameters of NeSy models. Learning the symbolic or logical structure of such models has, so far, received less attention. We introduce neurosymbolic decision trees (NDTs), as an extension of decision trees together with a novel NeSy structure learning algorithm, which we dub NeuID3. NeuID3 adapts the standard top-down induction of decision tree algorithms and combines it with a neural probabilistic logic representation, inherited from the DeepProbLog family of models. The key advantage of learning NDTs with NeuID3 is the support of both symbolic and subsymbolic data (such as images), and that they can exploit background knowledge during the induction of the tree structure, In our experimental evaluation we demonstrate the benefits of NeSys structure learning over more traditonal approaches such as purely data-driven learning with neural networks.

Figures (2)

• Figure 2: At the bottom we have a graphical representation of a probabilistic decision tree. Note that in this example, both $alarm$ literals, although in different branches, refer to the same test. Furthermore, we assign probability distributions over the classes $pos$ and $neg$ to each leaf node (cf. Equations \ref{['eq:lambdatwo']} and \ref{['eq:lambdathree']}). At the top we see the PDT expressed as a probabilistic logic program. We provide a translation algorithm in \ref{['ssec:nsdtc']} for the general neurosymbolic case.
• Figure 4: Figure shows how we define neural atoms in $\mathcal{T}^{nf}_{}$. A neural atom $rel_suit$ can learn relationships through a NN that receives both images as input. In our setting we define neural atoms only on both suits and ranks.