Improving Neural-based Classification with Logical Background Knowledge

TL;DR

A new neurosymbolic technique called semantic conditioning at inference is introduced, which only constrains the system during inference while leaving the training unaffected, and can be used to build more accurate neural-based systems with fewer resources while guaranteeing the semantic consistency of outputs.

Abstract

Neurosymbolic AI is a growing field of research aiming to combine neural networks learning capabilities with the reasoning abilities of symbolic systems. This hybridization can take many shapes. In this paper, we propose a new formalism for supervised multi-label classification with propositional background knowledge. We introduce a new neurosymbolic technique called semantic conditioning at inference, which only constrains the system during inference while leaving the training unaffected. We discuss its theoritical and practical advantages over two other popular neurosymbolic techniques: semantic conditioning and semantic regularization. We develop a new multi-scale methodology to evaluate how the benefits of a neurosymbolic technique evolve with the scale of the network. We then evaluate experimentally and compare the benefits of all three techniques across model scales on several datasets. Our results demonstrate that semantic conditioning at inference can be used to build more accurate neural-based systems with fewer resources while guaranteeing the semantic consistency of outputs.

Figures (3)

• Figure 1: Illustration of a neural-based classification system
• Figure 2: Results at the last epoch on all tasks and for all four techniques (error bars show the standard deviation over several seeds)
• Figure 3: Accuracy gap between sc and sci for all model sizes on all tasks

Theorems & Definitions (17)

• Definition 1
• Remark 1
• Definition 2
• Remark 2
• Definition 3
• Remark 3
• Definition 4
• Example 1
• Example 2
• Example 3