Symbol Grounding in Neuro-Symbolic AI: A Gentle Introduction to Reasoning Shortcuts

TL;DR

This work addresses symbol grounding in Neuro-Symbolic AI by focusing on Reasoning Shortcuts (RSs), where models achieve correct labels while grounding concepts incorrectly, harming interpretability and out-of-distribution reliability. It formalizes RSs through two complementary lenses: identifiability (when the ground-truth concept extractor is uniquely recoverable) and statistical learning (risk bounds under finite data), and introduces concept remapping as a core mechanism for RSs. The authors categorize mitigation and awareness strategies (concept supervision, multi-task learning, reconstruction, entropy, smoothing, disentanglement, and diffusion/ensembles) and relate their effectiveness to theoretical conditions like knowledge complexity and k-unambiguity, while also outlining diagnostics and practical trade-offs. Extensions to CBMs, foundation models, reinforcement learning, and imbalanced data are discussed, underscoring RSs as a general symbol-grounding challenge with broad implications for trustworthy NeSy systems. The work provides a unified framework and actionable guidance for reducing RS risk, improving grounding fidelity, and guiding future research toward reliable, interpretable NeSy AI.

Abstract

Neuro-symbolic (NeSy) AI aims to develop deep neural networks whose predictions comply with prior knowledge encoding, e.g. safety or structural constraints. As such, it represents one of the most promising avenues for reliable and trustworthy AI. The core idea behind NeSy AI is to combine neural and symbolic steps: neural networks are typically responsible for mapping low-level inputs into high-level symbolic concepts, while symbolic reasoning infers predictions compatible with the extracted concepts and the prior knowledge. Despite their promise, it was recently shown that - whenever the concepts are not supervised directly - NeSy models can be affected by Reasoning Shortcuts (RSs). That is, they can achieve high label accuracy by grounding the concepts incorrectly. RSs can compromise the interpretability of the model's explanations, performance in out-of-distribution scenarios, and therefore reliability. At the same time, RSs are difficult to detect and prevent unless concept supervision is available, which is typically not the case. However, the literature on RSs is scattered, making it difficult for researchers and practitioners to understand and tackle this challenging problem. This overview addresses this issue by providing a gentle introduction to RSs, discussing their causes and consequences in intuitive terms. It also reviews and elucidates existing theoretical characterizations of this phenomenon. Finally, it details methods for dealing with RSs, including mitigation and awareness strategies, and maps their benefits and limitations. By reformulating advanced material in a digestible form, this overview aims to provide a unifying perspective on RSs to lower the bar to entry for tackling them. Ultimately, we hope this overview contributes to the development of reliable NeSy and trustworthy AI models.

Figures (13)

• Figure 1: Reasoning shortcuts are failures of correct symbol grounding. Consider an autonomous driving car that must drive in compliance with traffic rules. (Left) The car learns to make correct predictions by leveraging background knowledge $\mathsf{K}\xspace_1$ in Task $1$, i.e., "if there is a pedestrian or a red light the car has to stop". However, in the process, it may incorrectly associate the concept of red_light with the presence of either pedestrians or red lights in the dashcam, resulting in what is known as a reasoning shortcut. (Right) Consequently, the learned concepts can lead to incorrect (and potentially catastrophic) predictions when the background knowledge changes. For example, $\mathsf{K}\xspace_2$ in Task $2$ introduces an exception in the presence of an emergency situation, i.e., "if there is an emergency and no pedestrians are on the street, the car may proceed".
• Figure 2: Schematic illustration of NeSy predictors. All of them employ a neural network for extracting concepts and prior knowledge $\mathsf{K}\xspace$. PNSPs (a) map inputs ${\bm{\mathrm{x}}}$ to (a distribution over) concepts ${\bm{\mathrm{c}}}$ and then uses probabilistic logic to infer labels ${\bm{\mathrm{y}}}$ consistent with $\mathsf{K}\xspace$. The SL (b) maps inputs to both concepts ${\bm{\mathrm{c}}}$ and labels ${\bm{\mathrm{y}}}$ and uses $\mathsf{K}\xspace$ to define a loss term penalizing inconsistent predictions. During training, ABL (c) abduces concepts from the ground-truth label ${\bm{\mathrm{y}}}$ using $\mathsf{K}\xspace$ and uses them as pseudo-labels to train the concept extractor. LTN (d) works similarly to PNSPs but uses fuzzy logic to make $\mathsf{K}\xspace$ differentiable. Lightning strikes indicate supervision.
• Figure 3: Left: truth table of the simplified prior knowledge $\mathsf{K}\xspace$ used in \ref{['ex:boia']}: the two concepts are interchangeable, in that as soon as one of them fires, the predictor infers the same (correct) action. Right: illustration of this RS: the model can confuse pedestrians and red lights with no drop in training loss.
• Figure 4: Left. The data generation process: $f^*$ maps inputs ${\bm{\mathrm{X}}}$ onto (distributions over) ground-truth concepts ${\bm{\mathrm{G}}}$, and $\beta^*$ maps these to (distributions over) labels ${\bm{\mathrm{Y}}}$. In a standard learning setting, only ${\bm{\mathrm{X}}}$ and ${\bm{\mathrm{Y}}}$ are observed (in light gray), whereas ${\bm{\mathrm{G}}}$ are latent. Right. A NeSy predictor maps inputs ${\bm{\mathrm{X}}}$ to concepts ${\bm{\mathrm{C}}}$ via a learned $f$ and infers labels ${\bm{\mathrm{Y}}}$ via an inference layer $\beta$. By \ref{['eq:definition-of-alpha']}, doing so entails a map $\alpha_f$ (in orange) from ground-truth concepts ${\bm{\mathrm{G}}}$ to learned concepts ${\bm{\mathrm{C}}}$.
• Figure 5: We connect concept extractors $f \in \mathcal{F}$ and maps $\alpha \in \mathcal{A}$ through \ref{['eq:definition-of-alpha']}. Here, we represent the space of learnable concept extractors $\mathcal{F}$ and the space $\mathcal{A}$ as a two-dimensional simplex. The three vertices correspond to the concept remappings $\mathrm{id}, {\bm{\mathrm{a}}}_1, {\bm{\mathrm{a}}}_2 \in \mathrm{Vert}(\mathcal{A})\xspace$. $\mathrm{id}$ is the intended solution, and the remaining mappings are deterministic RSs. From marconato2023not, any concept remapping ${\bm{\mathrm{a}}} \in \mathrm{Vert}(\mathcal{A})\xspace$ is in one-to-one correspondence with a concept extractor $f \in \mathcal{F}$ with domain restricted to $\mathrm{supp}({\bm{\mathrm{X}}})$ and, under \ref{['assu:concept-extractor']}, $f^*$ is in one-to-one correspondence with the identity function. However, non-deterministic concept remapping distributions $\alpha \in \mathcal{A} \setminus \mathrm{Vert}(\mathcal{A})\xspace$ can arise from several concept extractors in $\mathcal{F}$, here represented by the magenta colored subset mapping to one point in the simplex $\mathcal{A}$.

Theorems & Definitions (29)

• Example 1.1
• Example 2.1
• Example 2.2
• Definition 3.1: Reasoning Shortcut, Informal
• Example 3.2: RSs in MNIST-Add manhaeve2018deepproblog
• Example 3.3: RSs in BDD-OIA
• Example 3.4
• Example 3.5
• Example 3.6
• Theorem 4.4: Revisited from marconato2023neuro