Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Query languages for neural networks

Martin Grohe, Christoph Standke, Juno Steegmans, Jan Van den Bussche

TL;DR

This work proposes two declarative query paradigms for neural networks: FO(R), a constraint-based black-box language over the reals, and FO(SUM), a white-box language over weighted network structures with summation. It shows that FO(SUM) can express model-agnostic and nontrivial queries beyond FO(R_linear), and, crucially, for networks of fixed depth ℓ, FO(SUM) can express every FO(R_linear) query, effectively enabling SQL-like verification of neural models. The main technical contribution is a translation framework that encodes piecewise-linear network functions via hyperplane arrangements and cylindrical decompositions, then simulates FO(R_linear) queries within FO(SUM) through a sequence of FO(SUM) translations. Together, these results establish a formal, database-inspired foundation for explainability, verification, and management of neural networks using declarative query languages, with potential integration into practical analytics and tooling for ML deployment pipelines.

Abstract

We lay the foundations for a database-inspired approach to interpreting and understanding neural network models by querying them using declarative languages. Towards this end we study different query languages, based on first-order logic, that mainly differ in their access to the neural network model. First-order logic over the reals naturally yields a language which views the network as a black box; only the input--output function defined by the network can be queried. This is essentially the approach of constraint query languages. On the other hand, a white-box language can be obtained by viewing the network as a weighted graph, and extending first-order logic with summation over weight terms. The latter approach is essentially an abstraction of SQL. In general, the two approaches are incomparable in expressive power, as we will show. Under natural circumstances, however, the white-box approach can subsume the black-box approach; this is our main result. We prove the result concretely for linear constraint queries over real functions definable by feedforward neural networks with a fixed number of hidden layers and piecewise linear activation functions.

Query languages for neural networks

TL;DR

This work proposes two declarative query paradigms for neural networks: FO(R), a constraint-based black-box language over the reals, and FO(SUM), a white-box language over weighted network structures with summation. It shows that FO(SUM) can express model-agnostic and nontrivial queries beyond FO(R_linear), and, crucially, for networks of fixed depth ℓ, FO(SUM) can express every FO(R_linear) query, effectively enabling SQL-like verification of neural models. The main technical contribution is a translation framework that encodes piecewise-linear network functions via hyperplane arrangements and cylindrical decompositions, then simulates FO(R_linear) queries within FO(SUM) through a sequence of FO(SUM) translations. Together, these results establish a formal, database-inspired foundation for explainability, verification, and management of neural networks using declarative query languages, with potential integration into practical analytics and tooling for ML deployment pipelines.

Abstract

We lay the foundations for a database-inspired approach to interpreting and understanding neural network models by querying them using declarative languages. Towards this end we study different query languages, based on first-order logic, that mainly differ in their access to the neural network model. First-order logic over the reals naturally yields a language which views the network as a black box; only the input--output function defined by the network can be queried. This is essentially the approach of constraint query languages. On the other hand, a white-box language can be obtained by viewing the network as a weighted graph, and extending first-order logic with summation over weight terms. The latter approach is essentially an abstraction of SQL. In general, the two approaches are incomparable in expressive power, as we will show. Under natural circumstances, however, the white-box approach can subsume the black-box approach; this is our main result. We prove the result concretely for linear constraint queries over real functions definable by feedforward neural networks with a fixed number of hidden layers and piecewise linear activation functions.
Paper Structure (29 sections, 29 theorems, 8 equations, 4 figures)

This paper contains 29 sections, 29 theorems, 8 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Preliminaries on neural networks
  3. A black-box query language
  4. Weighted structures and $\textup{FO(SUM)}$
  5. White-box querying
  6. More examples
  7. Model-agnostic queries
  8. From $\textup{FO}({\mathbf R_{\rm lin}})$ to $\textup{FO(SUM)}$
  9. $\textup{FO(SUM)}$ translations
  10. Proof of Theorem \ref{['theormain']}
  11. Extension to multiple inputs
  12. Conclusion
  13. Proof of Lemma \ref{['lemintm']} for $m=1$ and $\ell = 2$
  14. Proof of Theorem \ref{['thm:fully_agnostic']}
  15. Addition of constants
  16. ...and 14 more sections

Key Result

Lemma 8

Let $m$ and $\ell$ be natural numbers. There exists an $\textup{FO(SUM)}$ term $t$ over $\Upsilon^{\mathrm{net}}(m,1)$ with $m$ additional pairs of weight constant symbols $\mathit{min}_i$ and $\mathit{max}_i$ for $i \in \{ 1, \dots, m\}$, such that for any network ${\mathcal{N}}$ in $\mathbf F(m,\e

Figures (4)

  • Figure 2: Area under a piece where both endpoints are positive. Using the additivity of the integration, all other cases can be reduced to this one.
  • Figure 3: Hyperplanes $h_1$ and $h_2$ define different section mappings (in red) above the cell $c_l$ in blue but define the same section mapping above the cells $c_r$
  • Figure 5: A $3$-dimensional $3$-cell and its base cell bellow it with some facets identified and highlighted in color, and all corners identified
  • Figure 6: Cylindrical decomposition of a $2$-D hyperplane arrangement with two 2-dimensional 2-cells $c_{\triangle}$ (in red) and $c_{\square}$ (in blue) highlighted with the identifiers of their corner points given.

Theorems & Definitions (41)

  • Example 1
  • Example 2
  • Example 3
  • Example 4
  • Remark 5
  • Example 6
  • Example 7
  • Lemma 8
  • Example 9
  • Theorem 10
  • ...and 31 more