Semialgebraic Neural Networks: From roots to representations

TL;DR

Semialgebraic Neural Networks (SANNs) introduce an architecture that integrates homotopy continuation with Inf-sup definable piecewise-polynomial networks to exactly represent bounded semialgebraic functions. By encoding graphs as kernels of ISD functions and solving a parameterized ODE that tracks roots along a homotopy, SANNs compute semialgebraic maps, including discontinuous ones, with a fixed evaluation procedure tied to an ODE solver. The authors prove expressivity for continuous and discontinuous cases, demonstrate a hand-crafted SANN for exact Jacobi inversion, and show training feasibility on linear systems with competitive performance. This work builds a theoretical and practical bridge between real algebraic geometry and neural computation, suggesting versatile, interpretable architectures for operator learning and polynomial-boundary problems.

Abstract

Many numerical algorithms in scientific computing -- particularly in areas like numerical linear algebra, PDE simulation, and inverse problems -- produce outputs that can be represented by semialgebraic functions; that is, the graph of the computed function can be described by finitely many polynomial equalities and inequalities. In this work, we introduce Semialgebraic Neural Networks (SANNs), a neural network architecture capable of representing any bounded semialgebraic function, and computing such functions up to the accuracy of a numerical ODE solver chosen by the programmer. Conceptually, we encode the graph of the learned function as the kernel of a piecewise polynomial selected from a class of functions whose roots can be evaluated using a particular homotopy continuation method. We show by construction that the SANN architecture is able to execute this continuation method, thus evaluating the learned semialgebraic function. Furthermore, the architecture can exactly represent even discontinuous semialgebraic functions by executing a continuation method on each connected component of the target function. Lastly, we provide example applications of these networks and show they can be trained with traditional deep-learning techniques.

Figures (7)

• Figure 1: Computing the Heaviside step function $F$ (plotted in red) using a homotopy method. The graph of $F$ is a subset of the kernel of a higher-dimensional piecewise polynomial $G$. In the upper row, we plot the surface of a homotopy $H$ such that $H(x,y,0)=0$ is trivial to solve (upper left), and $H(x,y,1)=G$ (upper right). The kernel of $H$ is shown with a green line (red points are also on top of the green line). The bottom row shows the projection of $H$ onto to $xy$-plane. We compute $F(x)=y$ by following the kernel of $H$ from time $t=0$ to $t=1$, keeping $x$ fixed. Although the kernel of $G$ contains points outside the graph of $F$ (the visible parts of the green lines), these points are never encountered when computing $F$ by following the homotopy. This process captures the discontinuity exactly, including the isolated point at $(0,0)$.
• Figure 2: Architecture diagram for a SANN. The SANN outputs $y$ as a semialgebraic function of input $x$. Vectors $z_j$ are the current values of an ODE at timestep $j$. The time-derivative $\dot{z}_j$ is computed using a neural network $\mathcal{N}$ capable of computing ISD piecewise polynomials. $\mathcal{N}$ accepts the current ODE state $(z_j, j/N)$, as well as recurrent input $x$. The output of $\mathcal{N}$ is a matrix $M$ and vector $b$, from which $\dot{z}_j$ is computed using $\hbox{clamp-sol}(M, b)$ ($\dot{z}_i = M^{-1} b$ in the common case). $\hbox{ODE-step}$ is a single update of a numerical ODE solver. Finally, $\Pi$ is a projection operation such that $y$ is the first $n$ components of $z$.
• Figure 3: Trajectories of a SANN that solves linear systems to machine precision. Given a matrix $X \in \mathbb{R}^{50 \times 50}$ and vector $b \in \mathbb{R}^{50}$, the network produces a vector $y \in \mathbb{R}^{50}$ such that $Xy = b$. Each line shows the trajectory of the output $y$ for a given input pair $(X, g)$ as the SANN's ODE is iteratively solved. At each timestep, the network performs a single Jacobi iteration update. The output of the network exactly matches the output of $100$ steps of Jacobi iteration.
• Figure 4: Encoding the graph of a semialgebraic function (red line) $F(x) = | x |$ as the kernel of a continuously differentiable piecewise polynomial (purple and yellow surface) $G(x, y) = ((x+y)(x-y))^2 + \max(0, -y)^2.$
• Figure 5: A $3 \times 3$ rectangular electrical network. Each edge in the graph represents a resistor with unknown conductance.

Theorems & Definitions (78)

• Definition 1: Inf-sup definable piecewise polynomials
• Definition 2: Locally ISD functions
• Definition 3: Basic semialgebraic set
• Definition 4: Semialgebraic set
• Definition 5: Semialgebraic function
• Example 6
• Proposition 7
• Corollary 8
• Definition 9
• Remark