Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving PDEs in One Shot via Fourier Features with Exact Analytical Derivatives

Antonin Sulc

TL;DR

Solving PDEs efficiently in high dimensions with mesh-free methods remains challenging for traditional PINNs and existing random-feature approaches. The paper introduces FastLSQ, which uses frozen sinusoidal random Fourier features and exact analytical derivatives to assemble the PDE operator in closed form, enabling a direct least-squares solve for linear PDEs and Newton-Raphson iterations for nonlinear PDEs. Key contributions include the cyclic derivative structure of sinusoidal features yielding $O(1)$ per-entry operator assembly, a true one-shot solver outperforming state-of-the-art baselines, and a robust Newton extension achieving $L^2$ errors from $10^{-8}$ to $10^{-9}$ on nonlinear PDEs in under 9 seconds across up to 6D problems. The results demonstrate a practical, deterministic alternative to PINNs with strong spectral accuracy and applicability to high-dimensional PDEs.

Abstract

Recent random feature methods for solving partial differential equations (PDEs) reduce computational cost compared to physics-informed neural networks (PINNs) but still rely on iterative optimization or expensive derivative computation. We observe that sinusoidal random Fourier features possess a cyclic derivative structure: the derivative of any order of $\sin(\mathbf{W}\cdot\mathbf{x}+b)$ is a single sinusoid with a monomial prefactor, computable in $O(1)$ operations. Alternative activations such as $\tanh$, used in prior one-shot methods like PIELM, lack this property: their higher-order derivatives grow as $O(2^n)$ terms, requiring automatic differentiation for operator assembly. We propose FastLSQ, which combines frozen random Fourier features with analytical operator assembly to solve linear PDEs via a single least-squares call, and extend it to nonlinear PDEs via Newton--Raphson iteration where each linearized step is a FastLSQ solve. On a benchmark of 17 PDEs spanning 1 to 6 dimensions, FastLSQ achieves relative $L^2$ errors of $10^{-7}$ in 0.07\,s on linear problems, three orders of magnitude more accurate and significantly faster than state-of-the-art iterative PINN solvers, and $10^{-8}$ to $10^{-9}$ on nonlinear problems via Newton iteration in under 9s.

Solving PDEs in One Shot via Fourier Features with Exact Analytical Derivatives

TL;DR

Solving PDEs efficiently in high dimensions with mesh-free methods remains challenging for traditional PINNs and existing random-feature approaches. The paper introduces FastLSQ, which uses frozen sinusoidal random Fourier features and exact analytical derivatives to assemble the PDE operator in closed form, enabling a direct least-squares solve for linear PDEs and Newton-Raphson iterations for nonlinear PDEs. Key contributions include the cyclic derivative structure of sinusoidal features yielding per-entry operator assembly, a true one-shot solver outperforming state-of-the-art baselines, and a robust Newton extension achieving errors from to on nonlinear PDEs in under 9 seconds across up to 6D problems. The results demonstrate a practical, deterministic alternative to PINNs with strong spectral accuracy and applicability to high-dimensional PDEs.

Abstract

Recent random feature methods for solving partial differential equations (PDEs) reduce computational cost compared to physics-informed neural networks (PINNs) but still rely on iterative optimization or expensive derivative computation. We observe that sinusoidal random Fourier features possess a cyclic derivative structure: the derivative of any order of is a single sinusoid with a monomial prefactor, computable in operations. Alternative activations such as , used in prior one-shot methods like PIELM, lack this property: their higher-order derivatives grow as terms, requiring automatic differentiation for operator assembly. We propose FastLSQ, which combines frozen random Fourier features with analytical operator assembly to solve linear PDEs via a single least-squares call, and extend it to nonlinear PDEs via Newton--Raphson iteration where each linearized step is a FastLSQ solve. On a benchmark of 17 PDEs spanning 1 to 6 dimensions, FastLSQ achieves relative errors of in 0.07\,s on linear problems, three orders of magnitude more accurate and significantly faster than state-of-the-art iterative PINN solvers, and to on nonlinear problems via Newton iteration in under 9s.
Paper Structure (25 sections, 2 theorems, 5 equations, 9 figures, 6 tables, 1 algorithm)

This paper contains 25 sections, 2 theorems, 5 equations, 9 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Key observation.
  3. Method
  4. Random Fourier Feature Approximation
  5. Exact Analytical Derivatives of Sinusoidal Features
  6. Solver Mode: Direct Linear Solve
  7. Newton--Raphson Extension for Nonlinear PDEs
  8. Experiments
  9. Setup
  10. Linear PDE Results
  11. Nonlinear PDE Results: Newton Solver Mode
  12. Ablation Study
  13. Nonlinear PDE Results: Regression Mode
  14. Gradient Accuracy
  15. Spectral Sensitivity Analysis
  16. ...and 10 more sections

Key Result

Corollary 2.1

The Laplacian satisfies $\Delta \phi_j = -\left\| \mathbf{W}_j \right\|^2 \sin(\mathbf{W}_j^\top \mathbf{x} + b_j)$; the biharmonic operator gives $\Delta^2 \phi_j = \left\| \mathbf{W}_j \right\|^4 \sin(\mathbf{W}_j^\top \mathbf{x} + b_j)$; and the advection operator yields $\mathbf{v} \cdot \nabla

Figures (9)

  • Figure 1: Architecture comparison. PINNs and RF-PDE require iterative optimization (top row). PIELM and Fast-LSQ both solve a linear system in one shot, but PIELM must invoke automatic differentiation to assemble the operator matrix from $\tanh$ features (middle row), whereas Fast-LSQ computes each operator entry analytically in $\mathcal{O}(1)$ (bottom row).
  • Figure 2: Newton convergence: NL-Poisson 2D (left) and Bratu 2D (right).
  • Figure 3: Newton convergence: Allen--Cahn 1D.
  • Figure 4: Spectral sensitivity: Poisson 5D (left) and Heat 5D (right).
  • Figure 5: Spectral sensitivity: Wave 1D (left) and Helmholtz 2D (right).
  • ...and 4 more figures

Theorems & Definitions (2)

  • Corollary 2.1: Common operators
  • Proposition 2.2: Contrast with $\tanh$