A tensor network formalism for neuro-symbolic AI

TL;DR

The paper presents a unified tensor-network framework that casts neural, probabilistic, and symbolic AI as contractions over structured representations. It introduces Computation-Activation Networks (CompActNets) and the tnreason library to model and train hybrid logical and probabilistic architectures, bridging propositional logic, graphical models, and neural decompositions. By representing logic with boolean tensors, probability with nonnegative tensors, and neural computations as tensor-network contractions, the approach enables scalable, exact or approximate inference via message passing and novel training schemes. This work advances explainable, verifiable neuro-symbolic AI with a practical toolkit for implementing and experimenting with hybrid models across domains.

Abstract

The unification of neural and symbolic approaches to artificial intelligence remains a central open challenge. In this work, we introduce a tensor network formalism, which captures sparsity principles originating in the different approaches in tensor decompositions. In particular, we describe a basis encoding scheme for functions and model neural decompositions as tensor decompositions. The proposed formalism can be applied to represent logical formulas and probability distributions as structured tensor decompositions. This unified treatment identifies tensor network contractions as a fundamental inference class and formulates efficiently scaling reasoning algorithms, originating from probability theory and propositional logic, as contraction message passing schemes. The framework enables the definition and training of hybrid logical and probabilistic models, which we call Hybrid Logic Network. The theoretical concepts are accompanied by the python library tnreason, which enables the implementation and practical use of the proposed architectures.

Figures (11)

• Figure 1: Sketch of the concepts in the neural, probabilistic and logical paradigms, which we define based on tensor network decompositions and contractions.
• Figure 2: Hypergraph to a $\mathrm{CP}$ format (see Example \ref{['exa:cpFormat']}). a) Node-centric design. b) Corresponding tensor network on the edges of the hypergraph.
• Figure 3: Hypergraph to a $\mathrm{TT}$ format (see Example \ref{['exa:ttFormat']}). a) Node-centric design. b) Corresponding tensor network on the edges of the hypergraph.
• Figure 4: Graphical depiction of a tensor network contraction with the open variables $X_{1},X_{3}$. Open variables are depicted by those without a dot at the end of the line.
• Figure 5: Decomposition of a probability distribution with independent variables (see Example \ref{['exa:coinTossHC']}). The independencies are captured by the elementary hypergraph a), whose edges contain single nodes. The corresponding tensor $\mathbb{P}\left[X_{[d]}\right]$ is then represented by a Markov network on the elementary hypergraph, where each factor is the marginal distribution of the corresponding variable as visualized in b).

Theorems & Definitions (55)

• Definition 1: Tensor
• Example 1: Delta tensor
• Definition 2: Tensor network
• Example 2: The $\mathrm{CP}$ format
• Example 3: The $\mathrm{TT}$ format
• Definition 3
• Example 4: Tensor product
• Definition 4
• Definition 5: One-hot encoding
• Definition 6: Basis encoding of maps between state sets