Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning

Haoyuan He, Wangzhou Dai, Ming Li

TL;DR

This paper identifies Implication Bias in differentiable logic losses used in Neuro-Symbolic learning and shows it can degrade performance when the knowledge base is incomplete or labeled data are scarce. It proposes Reduced Implication-bias Logic Loss (RILL), which downweights weak samples via hinge and $l_2$-based aggregators to reduce biased gradients without modifying the knowledge base. Empirical results on MNIST/FashionMNIST/CIFAR-10 demonstrate that RILL improves accuracy and stability under incomplete KB and limited supervision, achieving notable gains (e.g., CIFAR-10 with one labeled per class). The work offers a practical, model-agnostic approach to mitigating a pervasive bias in NeSy losses, enhancing robustness in real-world scenarios.

Abstract

Integrating logical reasoning and machine learning by approximating logical inference with differentiable operators is a widely used technique in Neuro-Symbolic systems. However, some differentiable operators could bring a significant bias during backpropagation and degrade the performance of Neuro-Symbolic learning. In this paper, we reveal that this bias, named \textit{Implication Bias} is common in loss functions derived from fuzzy logic operators. Furthermore, we propose a simple yet effective method to transform the biased loss functions into \textit{Reduced Implication-bias Logic Loss (RILL)} to address the above problem. Empirical study shows that RILL can achieve significant improvements compared with the biased logic loss functions, especially when the knowledge base is incomplete, and keeps more robust than the compared methods when labelled data is insufficient.

Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning

TL;DR

This paper identifies Implication Bias in differentiable logic losses used in Neuro-Symbolic learning and shows it can degrade performance when the knowledge base is incomplete or labeled data are scarce. It proposes Reduced Implication-bias Logic Loss (RILL), which downweights weak samples via hinge and -based aggregators to reduce biased gradients without modifying the knowledge base. Empirical results on MNIST/FashionMNIST/CIFAR-10 demonstrate that RILL improves accuracy and stability under incomplete KB and limited supervision, achieving notable gains (e.g., CIFAR-10 with one labeled per class). The work offers a practical, model-agnostic approach to mitigating a pervasive bias in NeSy losses, enhancing robustness in real-world scenarios.

Abstract

Integrating logical reasoning and machine learning by approximating logical inference with differentiable operators is a widely used technique in Neuro-Symbolic systems. However, some differentiable operators could bring a significant bias during backpropagation and degrade the performance of Neuro-Symbolic learning. In this paper, we reveal that this bias, named \textit{Implication Bias} is common in loss functions derived from fuzzy logic operators. Furthermore, we propose a simple yet effective method to transform the biased loss functions into \textit{Reduced Implication-bias Logic Loss (RILL)} to address the above problem. Empirical study shows that RILL can achieve significant improvements compared with the biased logic loss functions, especially when the knowledge base is incomplete, and keeps more robust than the compared methods when labelled data is insufficient.
Paper Structure (25 sections, 1 theorem, 19 equations, 9 figures, 2 tables)

This paper contains 25 sections, 1 theorem, 19 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Closed World Assumption
  5. Continuous-valued Logic
  6. Logic Loss in NeSy Systems
  7. Implication Bias
  8. Definition
  9. Case Study
  10. Analysis of Implication Bias
  11. Reduce Implication Bias Logic Loss (RILL)
  12. Empirical Study
  13. Setting Up
  14. Task 1 Addition Equations
  15. Task 2 Hierarchical Classification
  16. ...and 10 more sections

Key Result

Theorem 1

A logic loss $\ell_{\mathrm{logic}}$ that uses a $\delta$-confidence monotonic logic likelihood is implication biased.

Figures (9)

  • Figure 1: The implication bias is a tendency for NeSy systems to negate the premises of implication rules in order to increase consistency with the knowledge base.
  • Figure 2: Case study of implication bias. These logic rules are both satisfied their logical constraints.
  • Figure 3: Distribution of different samples given by the loss values, which are computed from rule $\text{Circle}(x)\rightarrow \text{Blue}(x)$.
  • Figure 4: Addition Equation Experiment, data are structured with equation constraints. All adopted equations in this experiment consist of four digits. For instance, in this picture, $1+5=06$.
  • Figure 5: Effect of incomplete knowledge base: In this case, models may not have access to all the relevant knowledge for making accurate reasoning.
  • ...and 4 more figures

Theorems & Definitions (7)

  • Definition 1: Implication Likelihood
  • Definition 2: Logic Likelihood
  • Definition 3: $\delta$-Confidence Monotonic
  • Definition 4: Logic Loss
  • Definition 5: Implication Biased
  • Theorem 1
  • Definition 6: Semantic Loss