Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recursive querying of neural networks via weighted structures

Martin Grohe, Christoph Standke, Juno Steegmans, Jan Van den Bussche

TL;DR

This work develops a logical framework for querying neural networks through weighted-structure logics. It introduces $FO(SUM)$ for weighted structures and extends it with recursion via $IFP(SUM)$, including a looser, update-friendly semantics and a normal-form result. A scalar fragment, $sIFP(SUM)$, achieves $PTIME$ data complexity and can express all PTIME model-agnostic queries on networks with polynomially bounded reduced weights, while simple $FO(\mathbf R_{\text{lin}},f)$ queries become NP-hard, highlighting expressivity limits. To overcome depth restrictions, the authors propose iterated $FO(SUM)$ transductions, which can express all $FO(\mathbf R,f)$ queries and reach computational completeness for rational weighted structures. Together, these results illuminate the trade-offs between expressive power and tractability in model-agnostic, logic-based querying of neural networks, and suggest directions for practical implementation and further theory in ML model verification and interpretation.

Abstract

Expressive querying of machine learning models - viewed as a form of intentional data - enables their verification and interpretation using declarative languages, thereby making learned representations of data more accessible. Motivated by the querying of feedforward neural networks, we investigate logics for weighted structures. In the absence of a bound on neural network depth, such logics must incorporate recursion; thereto we revisit the functional fixpoint mechanism proposed by Grädel and Gurevich. We adopt it in a Datalog-like syntax; we extend normal forms for fixpoint logics to weighted structures; and show an equivalent "loose" fixpoint mechanism that allows values of inductively defined weight functions to be overwritten. We propose a "scalar" restriction of functional fixpoint logic, of polynomial-time data complexity, and show it can express all PTIME model-agnostic queries over reduced networks with polynomially bounded weights. In contrast, we show that very simple model-agnostic queries are already NP-complete. Finally, we consider transformations of weighted structures by iterated transductions.

Recursive querying of neural networks via weighted structures

TL;DR

This work develops a logical framework for querying neural networks through weighted-structure logics. It introduces for weighted structures and extends it with recursion via , including a looser, update-friendly semantics and a normal-form result. A scalar fragment, , achieves data complexity and can express all PTIME model-agnostic queries on networks with polynomially bounded reduced weights, while simple queries become NP-hard, highlighting expressivity limits. To overcome depth restrictions, the authors propose iterated transductions, which can express all queries and reach computational completeness for rational weighted structures. Together, these results illuminate the trade-offs between expressive power and tractability in model-agnostic, logic-based querying of neural networks, and suggest directions for practical implementation and further theory in ML model verification and interpretation.

Abstract

Expressive querying of machine learning models - viewed as a form of intentional data - enables their verification and interpretation using declarative languages, thereby making learned representations of data more accessible. Motivated by the querying of feedforward neural networks, we investigate logics for weighted structures. In the absence of a bound on neural network depth, such logics must incorporate recursion; thereto we revisit the functional fixpoint mechanism proposed by Grädel and Gurevich. We adopt it in a Datalog-like syntax; we extend normal forms for fixpoint logics to weighted structures; and show an equivalent "loose" fixpoint mechanism that allows values of inductively defined weight functions to be overwritten. We propose a "scalar" restriction of functional fixpoint logic, of polynomial-time data complexity, and show it can express all PTIME model-agnostic queries over reduced networks with polynomially bounded weights. In contrast, we show that very simple model-agnostic queries are already NP-complete. Finally, we consider transformations of weighted structures by iterated transductions.
Paper Structure (24 sections, 19 theorems, 46 equations, 1 figure)

This paper contains 24 sections, 19 theorems, 46 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Related work
  3. Preliminaries
  4. FNNs
  5. Model-agnostic queries and $\mathsf{FO}(\mathbf R,f)$
  6. FO(SUM)
  7. FNNs as weighted structures
  8. Extending FO(SUM) with recursion
  9. The logic IFP(SUM)
  10. A loose semantics
  11. Comparing the two semantics
  12. A Normal Form
  13. A Polynomial Time Fragment
  14. Complexity of model-agnostic queries
  15. Iterated transductions
  16. ...and 9 more sections

Key Result

Theorem 3.4

Let $m$, $k$ and $\ell$ be natural numbers, and let $Q$ be a boolean query on $\mathbf K(m,1)$ with $k$ parameters, expressible in $\mathsf{FO}(\mathbf R_{\text{\upshape lin}},f)$. There exists an FO(SUM) sentence $\psi$ over vocabulary $(E,\mathrm{In},\mathrm{Out},b,w,\mathit{val})$ that expresses

Figures (1)

  • Figure 1: Splitting (a) an edge with large weight $a\in\mathbb N$ into (b) $a$ internal nodes connected by edges of weight $1$. Reducing the net in (b) yields the net (c)

Theorems & Definitions (45)

  • Example 3.1
  • Example 3.2
  • Example 3.3
  • Theorem 3.4: ql4nn
  • Example 4.1
  • Example 4.2
  • Definition 4.3
  • Example 4.4
  • Proposition 4.5
  • proof
  • ...and 35 more