Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parameter-Minimal Neural DE Solvers via Horner Polynomials

T. Matulić, D. Seršić

TL;DR

This work tackles the efficiency of neural DE solvers by restricting the hypothesis class to Horner-factorized polynomials, and by hard-embedding initial conditions into the architecture to eliminate penalty tuning. The Horner network provides a differentiable, low-parameter trial solution, with a spline-like extension enabling piecewise, continuous approximations that maintain a small parameter footprint. On canonical ODE benchmarks and a heat PDE, Horner networks with as few as 10–13 learnable parameters achieve state-competitive accuracy for the solution and its derivatives, outperforming compact MLP and SIREN baselines. The results demonstrate a practical accuracy–parameter trade-off and open avenues for scalable, interpretable, and resource-efficient scientific modeling, including extensions to higher-dimensional PDEs and boundary-conditioned architectures.

Abstract

We propose a parameter-minimal neural architecture for solving differential equations by restricting the hypothesis class to Horner-factorized polynomials, yielding an implicit, differentiable trial solution with only a small set of learnable coefficients. Initial conditions are enforced exactly by construction by fixing the low-order polynomial degrees of freedom, so training focuses solely on matching the differential-equation residual at collocation points. To reduce approximation error without abandoning the low-parameter regime, we introduce a piecewise ("spline-like") extension that trains multiple small Horner models on subintervals while enforcing continuity (and first-derivative continuity) at segment boundaries. On illustrative ODE benchmarks and a heat-equation example, Horner networks with tens (or fewer) parameters accurately match the solution and its derivatives and outperform small MLP and sinusoidal-representation baselines under the same training settings, demonstrating a practical accuracy-parameter trade-off for resource-efficient scientific modeling.

Parameter-Minimal Neural DE Solvers via Horner Polynomials

TL;DR

This work tackles the efficiency of neural DE solvers by restricting the hypothesis class to Horner-factorized polynomials, and by hard-embedding initial conditions into the architecture to eliminate penalty tuning. The Horner network provides a differentiable, low-parameter trial solution, with a spline-like extension enabling piecewise, continuous approximations that maintain a small parameter footprint. On canonical ODE benchmarks and a heat PDE, Horner networks with as few as 10–13 learnable parameters achieve state-competitive accuracy for the solution and its derivatives, outperforming compact MLP and SIREN baselines. The results demonstrate a practical accuracy–parameter trade-off and open avenues for scalable, interpretable, and resource-efficient scientific modeling, including extensions to higher-dimensional PDEs and boundary-conditioned architectures.

Abstract

We propose a parameter-minimal neural architecture for solving differential equations by restricting the hypothesis class to Horner-factorized polynomials, yielding an implicit, differentiable trial solution with only a small set of learnable coefficients. Initial conditions are enforced exactly by construction by fixing the low-order polynomial degrees of freedom, so training focuses solely on matching the differential-equation residual at collocation points. To reduce approximation error without abandoning the low-parameter regime, we introduce a piecewise ("spline-like") extension that trains multiple small Horner models on subintervals while enforcing continuity (and first-derivative continuity) at segment boundaries. On illustrative ODE benchmarks and a heat-equation example, Horner networks with tens (or fewer) parameters accurately match the solution and its derivatives and outperform small MLP and sinusoidal-representation baselines under the same training settings, demonstrating a practical accuracy-parameter trade-off for resource-efficient scientific modeling.
Paper Structure (30 sections, 43 equations, 9 figures, 2 tables)

This paper contains 30 sections, 43 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Notation and Benchmark Problems
  3. Problem setup
  4. Collocation data
  5. Benchmark problems
  6. Baseline Solvers: MLP and SIREN Networks
  7. Residual loss with soft initial-condition penalties
  8. Baseline architectures and training protocol
  9. Baseline 1 (MLP--Leaky ReLU).
  10. Baseline 2 (MLP--sigmoid).
  11. Baseline 3 (SIREN).
  12. Results and discussion
  13. Motivation: Linear Regression Viewpoint
  14. Linear ODE with constant coefficients
  15. Polynomial regression model
  16. ...and 15 more sections

Figures (9)

  • Figure 1: Type A benchmark: baseline network predictions for the solution and its derivatives. Rows correspond to architectures (MLP with Leaky ReLU, MLP with sigmoid, and SIREN), while columns show $x(t)$, $x'(t)$, and $x"(t)$. Each panel compares the learned output against the analytical reference.
  • Figure 2: Type A: polynomial regression (degree $m=15$) versus the analytical solution on $I=[0,4]$.
  • Figure 3: Matched forcing: polynomial regression (degree $m=15$) versus the analytical solution on $I=[0,4]$.
  • Figure 4: Type C: polynomial regression (degree $m=15$) versus the analytical solution on $I=[0,3]$.
  • Figure 5: Horner-based architecture used as a parameter-minimal neural solver.
  • ...and 4 more figures