Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Symbolic Logical Rule Learner for Interpretable Learning

Bowen Wei, Ziwei Zhu

TL;DR

The paper tackles the challenge of interpretable learning while preserving predictive accuracy by introducing the Normal Form Rule Learner (NFRL), a discrete neural framework that learns CNF and DNF rules. It combines two Normal Form Layers (NFLs) with adaptive AND/OR neurons, a Negation Layer for input negations, and a Normal Form Constraint (NFC) to efficiently manage connections; optimization is performed end-to-end via the Straight-Through Estimator on binary-valued neurons with $\pm1$ states. Empirical results on 11 datasets show NFRL surpassing 12 baselines in F1 score, rule quality (diversity, coverage, single-rule accuracy), and training efficiency, while yielding concise, interpretable rule sets; the authors also provide extensive ablations and simulation studies to validate the components. The work advances scalable, interpretable rule learning by fusing symbolic CNF/DNF reasoning with neural optimization, enabling accurate, transparent decision-making suitable for high-stakes domains, with code and data available for reproducibility.

Abstract

Rule-based neural networks stand out for enabling interpretable classification by learning logical rules for both prediction and interpretation. However, existing models often lack flexibility due to the fixed model structure. Addressing this, we introduce the Normal Form Rule Learner (NFRL) algorithm, leveraging a selective discrete neural network, that treat weight parameters as hard selectors, to learn rules in both Conjunctive Normal Form (CNF) and Disjunctive Normal Form (DNF) for enhanced accuracy and interpretability. Instead of adopting a deep, complex structure, the NFRL incorporates two specialized Normal Form Layers (NFLs) with adaptable AND/OR neurons, a Negation Layer for input negations, and a Normal Form Constraint (NFC) to streamline neuron connections. We also show the novel network architecture can be optimized using adaptive gradient update together with Straight-Through Estimator to overcome the gradient vanishing challenge. Through extensive experiments on 11 datasets, NFRL demonstrates superior classification performance, quality of learned rules, efficiency and interpretability compared to 12 state-of-the-art alternatives. Code and data are available at \url{https://anonymous.4open.science/r/NFRL-27B4/}.

Neural Symbolic Logical Rule Learner for Interpretable Learning

TL;DR

The paper tackles the challenge of interpretable learning while preserving predictive accuracy by introducing the Normal Form Rule Learner (NFRL), a discrete neural framework that learns CNF and DNF rules. It combines two Normal Form Layers (NFLs) with adaptive AND/OR neurons, a Negation Layer for input negations, and a Normal Form Constraint (NFC) to efficiently manage connections; optimization is performed end-to-end via the Straight-Through Estimator on binary-valued neurons with states. Empirical results on 11 datasets show NFRL surpassing 12 baselines in F1 score, rule quality (diversity, coverage, single-rule accuracy), and training efficiency, while yielding concise, interpretable rule sets; the authors also provide extensive ablations and simulation studies to validate the components. The work advances scalable, interpretable rule learning by fusing symbolic CNF/DNF reasoning with neural optimization, enabling accurate, transparent decision-making suitable for high-stakes domains, with code and data available for reproducibility.

Abstract

Rule-based neural networks stand out for enabling interpretable classification by learning logical rules for both prediction and interpretation. However, existing models often lack flexibility due to the fixed model structure. Addressing this, we introduce the Normal Form Rule Learner (NFRL) algorithm, leveraging a selective discrete neural network, that treat weight parameters as hard selectors, to learn rules in both Conjunctive Normal Form (CNF) and Disjunctive Normal Form (DNF) for enhanced accuracy and interpretability. Instead of adopting a deep, complex structure, the NFRL incorporates two specialized Normal Form Layers (NFLs) with adaptable AND/OR neurons, a Negation Layer for input negations, and a Normal Form Constraint (NFC) to streamline neuron connections. We also show the novel network architecture can be optimized using adaptive gradient update together with Straight-Through Estimator to overcome the gradient vanishing challenge. Through extensive experiments on 11 datasets, NFRL demonstrates superior classification performance, quality of learned rules, efficiency and interpretability compared to 12 state-of-the-art alternatives. Code and data are available at \url{https://anonymous.4open.science/r/NFRL-27B4/}.
Paper Structure (34 sections, 12 equations, 10 figures, 8 tables)

This paper contains 34 sections, 12 equations, 10 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Rule Learning Methods
  4. Rule Learning Neural Networks
  5. Binarized Neural Network
  6. Preliminaries
  7. Problem Formulation
  8. Feature Binarization
  9. Normal Form Rules as Model Interpretation
  10. Method
  11. Overall Structure
  12. Normal Form Layer (NFL)
  13. Negation Layer
  14. Optimization
  15. Normal Form Constraint (NFC)
  16. ...and 19 more sections

Figures (10)

  • Figure 1: Overall Structure.
  • Figure 2: Comparison of Various Metrics. (a) Diversity. (b) Coverage. (c) Accuracy. (d) Average Rule Length. (e) Comparison of Computational Time.
  • Figure 3: Comparison of F1 Scores for NFRL and RRL with Different Numbers of Generated Rules.
  • Figure 4: Performance under different Architecture
  • Figure 5: Performance under different L2 regularization
  • ...and 5 more figures