Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Incorporating Expert Rules into Neural Networks in the Framework of Concept-Based Learning

Andrei V. Konstantinov, Lev V. Utkin

TL;DR

This work addresses integrating expert rules into neural networks within concept-based learning (CBL). By representing feasible concept distributions as either a convex polytope (with vertices or faces) or a set of linear inequalities derived from the rules, the authors design neural heads that guarantee rule-consistent concept probabilities. They propose four neural-head strategies—Base, Admissible State, Vertex-based, and Constraints heads—along with state-space reduction to scale to larger rule sets. Numerical experiments on synthetic data and a modified MNIST task demonstrate improved performance under partial labeling, validating the approach as a practical bridge between inductive and deductive learning with interpretable predictions.

Abstract

A problem of incorporating the expert rules into machine learning models for extending the concept-based learning is formulated in the paper. It is proposed how to combine logical rules and neural networks predicting the concept probabilities. The first idea behind the combination is to form constraints for a joint probability distribution over all combinations of concept values to satisfy the expert rules. The second idea is to represent a feasible set of probability distributions in the form of a convex polytope and to use its vertices or faces. We provide several approaches for solving the stated problem and for training neural networks which guarantee that the output probabilities of concepts would not violate the expert rules. The solution of the problem can be viewed as a way for combining the inductive and deductive learning. Expert rules are used in a broader sense when any logical function that connects concepts and class labels or just concepts with each other can be regarded as a rule. This feature significantly expands the class of the proposed results. Numerical examples illustrate the approaches. The code of proposed algorithms is publicly available.

Incorporating Expert Rules into Neural Networks in the Framework of Concept-Based Learning

TL;DR

This work addresses integrating expert rules into neural networks within concept-based learning (CBL). By representing feasible concept distributions as either a convex polytope (with vertices or faces) or a set of linear inequalities derived from the rules, the authors design neural heads that guarantee rule-consistent concept probabilities. They propose four neural-head strategies—Base, Admissible State, Vertex-based, and Constraints heads—along with state-space reduction to scale to larger rule sets. Numerical experiments on synthetic data and a modified MNIST task demonstrate improved performance under partial labeling, validating the approach as a practical bridge between inductive and deductive learning with interpretable predictions.

Abstract

A problem of incorporating the expert rules into machine learning models for extending the concept-based learning is formulated in the paper. It is proposed how to combine logical rules and neural networks predicting the concept probabilities. The first idea behind the combination is to form constraints for a joint probability distribution over all combinations of concept values to satisfy the expert rules. The second idea is to represent a feasible set of probability distributions in the form of a convex polytope and to use its vertices or faces. We provide several approaches for solving the stated problem and for training neural networks which guarantee that the output probabilities of concepts would not violate the expert rules. The solution of the problem can be viewed as a way for combining the inductive and deductive learning. Expert rules are used in a broader sense when any logical function that connects concepts and class labels or just concepts with each other can be regarded as a rule. This feature significantly expands the class of the proposed results. Numerical examples illustrate the approaches. The code of proposed algorithms is publicly available.
Paper Structure (20 sections, 1 theorem, 42 equations, 6 figures, 1 table)

This paper contains 20 sections, 1 theorem, 42 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Background
  4. Expert rules and concepts
  5. Problem statement
  6. Joint distribution
  7. Reduction to a linear constraint
  8. Probabilistic approach
  9. Solution set as a polytope
  10. Linear inequality system
  11. Neural network and expert rules
  12. Base head
  13. Admissible state head
  14. Vertex-based head
  15. Constraints head
  16. ...and 5 more sections

Key Result

Theorem 1

Given a rule $\mathcal{R}$ in CNF, consisting of $b$ clauses: the constraint on the expert rule probability is equivalent to intersection of clause constraints:

Figures (6)

  • Figure 1: The Base Head (the top picture) and the AS-Head (the bottom picture) approaches
  • Figure 2: The Vetex Head (the left picture) and Constraints Head (the right picture) approaches
  • Figure 3: Concepts used in the toy synthetic example
  • Figure 4: Predicted probabilities with the implication rule
  • Figure 5: Predicted probabilities with the “ if and only if” rule
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof