Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts

Emanuele Marconato, Samuele Bortolotti, Emile van Krieken, Antonio Vergari, Andrea Passerini, Stefano Teso

TL;DR

This work tackles the problem that Neuro-Symbolic predictors can learn concepts with unintended semantics (Reasoning Shortcuts) while still achieving correct labels under the encoded knowledge. It introduces bears, an ensemble-based approach that calibrates concept-level uncertainty to identify RS-affected concepts without relying on costly dense annotations. The method is analyzed theoretically—via entropy maximization and convex decompositions of concept mappings—and empirically validated across multiple NeSy architectures and datasets, showing substantial improvements in RS-awareness and enabling more effective active learning for mitigation. By shifting from mitigation to awareness, bears enhances reliability and interpretability of NeSy systems in high-stakes tasks while maintaining strong predictive performance.

Abstract

Neuro-Symbolic (NeSy) predictors that conform to symbolic knowledge - encoding, e.g., safety constraints - can be affected by Reasoning Shortcuts (RSs): They learn concepts consistent with the symbolic knowledge by exploiting unintended semantics. RSs compromise reliability and generalization and, as we show in this paper, they are linked to NeSy models being overconfident about the predicted concepts. Unfortunately, the only trustworthy mitigation strategy requires collecting costly dense supervision over the concepts. Rather than attempting to avoid RSs altogether, we propose to ensure NeSy models are aware of the semantic ambiguity of the concepts they learn, thus enabling their users to identify and distrust low-quality concepts. Starting from three simple desiderata, we derive bears (BE Aware of Reasoning Shortcuts), an ensembling technique that calibrates the model's concept-level confidence without compromising prediction accuracy, thus encouraging NeSy architectures to be uncertain about concepts affected by RSs. We show empirically that bears improves RS-awareness of several state-of-the-art NeSy models, and also facilitates acquiring informative dense annotations for mitigation purposes.

BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts

TL;DR

This work tackles the problem that Neuro-Symbolic predictors can learn concepts with unintended semantics (Reasoning Shortcuts) while still achieving correct labels under the encoded knowledge. It introduces bears, an ensemble-based approach that calibrates concept-level uncertainty to identify RS-affected concepts without relying on costly dense annotations. The method is analyzed theoretically—via entropy maximization and convex decompositions of concept mappings—and empirically validated across multiple NeSy architectures and datasets, showing substantial improvements in RS-awareness and enabling more effective active learning for mitigation. By shifting from mitigation to awareness, bears enhances reliability and interpretability of NeSy systems in high-stakes tasks while maintaining strong predictive performance.

Abstract

Neuro-Symbolic (NeSy) predictors that conform to symbolic knowledge - encoding, e.g., safety constraints - can be affected by Reasoning Shortcuts (RSs): They learn concepts consistent with the symbolic knowledge by exploiting unintended semantics. RSs compromise reliability and generalization and, as we show in this paper, they are linked to NeSy models being overconfident about the predicted concepts. Unfortunately, the only trustworthy mitigation strategy requires collecting costly dense supervision over the concepts. Rather than attempting to avoid RSs altogether, we propose to ensure NeSy models are aware of the semantic ambiguity of the concepts they learn, thus enabling their users to identify and distrust low-quality concepts. Starting from three simple desiderata, we derive bears (BE Aware of Reasoning Shortcuts), an ensembling technique that calibrates the model's concept-level confidence without compromising prediction accuracy, thus encouraging NeSy architectures to be uncertain about concepts affected by RSs. We show empirically that bears improves RS-awareness of several state-of-the-art NeSy models, and also facilitates acquiring informative dense annotations for mitigation purposes.
Paper Structure (47 sections, 10 theorems, 55 equations, 17 figures, 12 tables, 1 algorithm)

This paper contains 47 sections, 10 theorems, 55 equations, 17 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation.
  4. Reasoning Shortcuts.
  5. From Mitigation to Awareness
  6. Awareness Via Entropy Maximization
  7. Maximizing Entropy
  8. RS-Awareness with bears
  9. bears through a Bayesian Lens
  10. Active Learning with Dense Annotations
  11. Benefits and Limitations
  12. Empirical Analysis
  13. Related Work
  14. Conclusion
  15. Implementation Details
  16. ...and 32 more sections

Key Result

Lemma 1

For any $p({\bm{\mathrm{C}}} \mid {\bm{\mathrm{G}}})$, there exists at least one vector ${\bm{\mathrm{\omega}}}$ such that the following holds: where ${\bm{\mathrm{\omega}}} \ge 0$, $\lVert{\bm{\mathrm{\omega}}}\rVert_1 = 1$. Crucially, under invertibility (A1) and determinism (A2), if $p_\theta({\bm{\mathrm{C}}} \mid {\bm{\mathrm{G}}})$ is optimal (D2), eq:whatever holds even if we replace $\mat

Figures (17)

  • Figure 1: bears lessens overconfidence due to reasoning shortcuts.Left: In the BDD-OIAautonomous driving task xu2020boiasawada2022concept, NeSy predictors can attain high accuracy and comply with the knowledge even when confusing the concepts of pedestrian (${\tt ped}$) and red light (${\tt red}$) marconato2023not. Middle: State-of-the-art NeSy architectures predict concepts affected by RSs with high confidence, making it impossible to discriminate between reliable and unreliable concept predictions. Right: bears encourages them to allocate probability to conflicting concept maps, substantially lessening overconfidence.
  • Figure 2: Data generating process. The (unobserved) ground-truth concepts ${\bm{\mathrm{G}}}$ cause the inputs ${\bm{\mathrm{X}}}$ which cause the labels ${\bm{\mathrm{Y}}}$ (in black). A NeSy predictor learns to map inputs ${\bm{\mathrm{X}}}$ to concepts ${\bm{\mathrm{C}}}$ (in blue), which ideally should match the concepts ${\bm{\mathrm{G}}}$ that caused ${\bm{\mathrm{X}}}$. The maps $f$ and $\beta_\mathsf{K}\xspace$ from assumptions A1 and A2 in \ref{['sec:method']} are shown in red.
  • Figure 3: Per-concept entropy shows bears is more uncertain about concepts affected by RS on MNIST-Half compared to regular DPL and alternative uncertainty calibration methods. SL and LTN show similar trends, see \ref{['sec:all-results']}. Importantly, these improvements do not require concept annotations.
  • Figure 4: bears allows selecting informative concept annotations faster. A substantial improvement in concept accuracy is achieved by performing active learning guided by RS-aware concept uncertainty (DPL+bears) with respect to plain concept uncertainty (DPL) and random selection.
  • Figure 5: An example of a test sample for the Kandinsky task. At inference time, the NeSy model has to choose according to the previous two images the third that completes the pattern. For each image, the model computes a series of predicates, like $\texttt{same\_cs}$, $\texttt{same\_ss}$, and so on. In this case, the first two images have different colors, so the model should pick the first option.
  • ...and 12 more figures

Theorems & Definitions (19)

  • Example 1
  • Example 2
  • Example 3
  • Lemma 1
  • Example 4
  • Proposition 2
  • Proposition 3
  • Lemma 4
  • Theorem 5
  • Proposition 6
  • ...and 9 more