Fast training of accurate physics-informed neural networks without gradient descent

TL;DR

The paper tackles two main obstacles in solving time-dependent PDEs with PINNs: optimizers dealing with high-dimensional, multi-objective loss landscapes and the non-causal treatment of time. It introduces Frozen-PINN, which employs space–time separation with frozen random-space bases and time-varying output weights, decouples loss terms, and evolves time through adaptive ODE solvers rather than gradient descent. By sampling hidden-layer parameters via ELM or SWIM, converting PDEs to ODEs for the output coefficients, and optionally using a boundary-compliant layer or augmented ODEs, the method achieves temporally causal solutions with dramatically faster training and higher accuracy across eight challenging PDE benchmarks, including high advection, shocks, nonlinearities, and very high dimensions. An SVD layer further compresses the system and speeds up computation, enabling CPU-based training to rival or exceed GPU-based PINNs in many cases. The approach demonstrates a potential paradigm shift in neural PDE solvers, delivering quickly trainable, highly accurate, and inherently causal solvers while offering robust extensibility for future complex systems such as Navier–Stokes.

Abstract

Solving time-dependent Partial Differential Equations (PDEs) is one of the most critical problems in computational science. While Physics-Informed Neural Networks (PINNs) offer a promising framework for approximating PDE solutions, their accuracy and training speed are limited by two core barriers: gradient-descent-based iterative optimization over complex loss landscapes and non-causal treatment of time as an extra spatial dimension. We present Frozen-PINN, a novel PINN based on the principle of space-time separation that leverages random features instead of training with gradient descent, and incorporates temporal causality by construction. On eight PDE benchmarks, including challenges such as extreme advection speeds, shocks, and high dimensionality, Frozen-PINNs achieve superior training efficiency and accuracy over state-of-the-art PINNs, often by several orders of magnitude. Our work addresses longstanding training and accuracy bottlenecks of PINNs, delivering quickly trainable, highly accurate, and inherently causal PDE solvers, a combination that prior methods could not realize. Our approach challenges the reliance of PINNs on stochastic gradient-descent-based methods and specialized hardware, leading to a paradigm shift in PINN training and providing a challenging benchmark for the community.

Figures (23)

• Figure 1: Core ideas of Frozen-PINNs: (1) Space–time separation: For $x \in \mathbb{R}^d$, spatial bases $\phi_i = \sigma(w_i \cdot x + b_i)$ with $\sigma = \tanh$, $w_i \in \mathbb{R}^d$, $b_i \in \mathbb{R}$ are sampled and frozen (shown for $d = 1$); output layer parameters $c_i(t)$ are evolved via ODEs. Each pair $(\phi_i, c_i)$ is color-matched. (2) Loss decoupling: PDE, boundary, and initial condition losses $L_{\text{PDE}}, L_{\text{BC}}, L_{\text{IC}}$ are optimized independently.
• Figure 2: Sampling in Frozen-PINNs: (Left): SWIM (data-driven, places bases with steep gradients near regions with shocks) vs. (Right): ELM (data-agnostic, no control over basis placement).
• Figure 3: Architecture of Frozen-PINNs trained with a gradient-descent-free training algorithm.
• Figure 4: Illustration of experimental results for the advection equation: (Left): high advection speeds - effect of advection coefficient $\beta$ on the test error for different PDE solvers, (Middle): fast convergence - with $\beta = 10$, Frozen PINNs achieve exponential decay in test error as indicated by the reference dotted line, while standard PINNs display plateaued error decay despite increasing number of basis functions (hidden layer size), (Right): long time simulation - Slow error growth with time.
• Figure 5: Constructing useful Frozen-PINN-swim bases. (Left): shock-aware sampling (Burgers, (\ref{['ss_burgers']})) and (Right): direction-aware bases (reaction–diffusion, \ref{['nonlin_reaction_diff']}).