Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Extracting Formulae in Many-Valued Logic from Deep Neural Networks

Yani Zhang, Helmut Bölcskei

TL;DR

The paper establishes a rigorous link between deep ReLU networks and many‑valued MV logic via McNaughton functions, proving a two‑way correspondence between integer‑weighted ReLU networks and MV terms. It then provides a constructive algorithm to extract MV formulae from trained networks, and extends the approach to rational and real weights (Rational Łukasiewicz logic and RMV logic) with suitable adaptations. A key finding is that deep architectures yield substantially shorter MV terms than shallow or noncompositional methods, especially for multi‑dimensional McNaughton functions, illustrating a formal advantage of depth in encoding logical structure. Practically, this framework enables interpretable logical descriptions of data representations learned by deep nets and broadens the bridge between neural computation and formal logic.

Abstract

We propose a new perspective on deep ReLU networks, namely as circuit counterparts of Lukasiewicz infinite-valued logic -- a many-valued (MV) generalization of Boolean logic. An algorithm for extracting formulae in MV logic from deep ReLU networks is presented. As the algorithm applies to networks with general, in particular also real-valued, weights, it can be used to extract logical formulae from deep ReLU networks trained on data.

Extracting Formulae in Many-Valued Logic from Deep Neural Networks

TL;DR

The paper establishes a rigorous link between deep ReLU networks and many‑valued MV logic via McNaughton functions, proving a two‑way correspondence between integer‑weighted ReLU networks and MV terms. It then provides a constructive algorithm to extract MV formulae from trained networks, and extends the approach to rational and real weights (Rational Łukasiewicz logic and RMV logic) with suitable adaptations. A key finding is that deep architectures yield substantially shorter MV terms than shallow or noncompositional methods, especially for multi‑dimensional McNaughton functions, illustrating a formal advantage of depth in encoding logical structure. Practically, this framework enables interpretable logical descriptions of data representations learned by deep nets and broadens the bridge between neural computation and formal logic.

Abstract

We propose a new perspective on deep ReLU networks, namely as circuit counterparts of Lukasiewicz infinite-valued logic -- a many-valued (MV) generalization of Boolean logic. An algorithm for extracting formulae in MV logic from deep ReLU networks is presented. As the algorithm applies to networks with general, in particular also real-valued, weights, it can be used to extract logical formulae from deep ReLU networks trained on data.
Paper Structure (17 sections, 8 theorems, 69 equations, 6 figures)

This paper contains 17 sections, 8 theorems, 69 equations, 6 figures.

Table of Contents

  1. Introduction and Contributions
  2. Boolean Logic and MV Logic
  3. Extraction of formulae in MV logic from ReLU networks
  4. Existing results on MV term extraction from McNaughton functions
  5. Deep networks yield shorter formulae
  6. Analytical characterization of MV term lengths
  7. Expressive power of deep networks
  8. The multi-dimensional case
  9. Extensions to the Rational and Real Cases
  10. The Rational Case
  11. The Real Case
  12. MV algebra
  13. ReLU networks
  14. Proof of Lemma \ref{['lem:extractmv']}
  15. Proof of Lemma \ref{['lem:extractmvR']}
  16. ...and 2 more sections

Key Result

Theorem 3.1

Consider the MV algebra $\mathcal{I}$ in Definition def:MV01. Let $n\in \mathbb{N}$. For a function $f: [0,1]^n \rightarrow [0,1]$ to have a corresponding MV term $\tau$ such that the associated term function $\tau ^{\mathcal{I}}$ satisfies $\tau ^{\mathcal{I}} = f$ on $[0,1]^n$, it is necessary and Functions satisfying these conditions are called McNaughton functions.

Figures (6)

  • Figure 1: Left: The function $g$, right: The function $g_2$.
  • Figure 2: Length of the MV terms extracted from $g_s$ using different methods.
  • Figure 3: Lengths of MV terms extracted by different methods, the figure on the right depicts the same results as that on the left, but without the Shallow NN method.
  • Figure 4: Average number of breakpoints of randomly generated MV terms.
  • Figure 5: Lengths of MV terms extracted by different methods.
  • ...and 1 more figures

Theorems & Definitions (15)

  • Definition 2.1
  • Theorem 3.1: mcnaughton1951theorem
  • Theorem 3.2
  • Lemma 3.3: rose1958mundici1994constructive
  • Lemma 3.4
  • Definition 6.1
  • Theorem 6.2
  • Definition 6.3
  • Lemma 6.4: di2014lukasiewicz
  • Definition A.1: cignoli2013algebraic
  • ...and 5 more