Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction

TL;DR

Scale-PINN blends the residual-correction principles of iterative solvers with physics-informed neural networks by embedding a sequential correction loss that applies a residual-smoothing operator $\mathcal{P}_{\alpha}=(I-\alpha^2\nabla^2)$ to the update $\mathcal{F}=f(\cdot;\boldsymbol{w}^k)-f(\cdot;\boldsymbol{w}^{k-1})$, yielding fast, stable convergence for diverse PDEs. The method delivers sub-minute training on challenging Navier–Stokes problems and demonstrates strong accuracy across benchmark PDEs (KS, Grey–Scott, KdV, Allen–Cahn) while reducing reliance on curriculum or data supervision. This work provides a practical, scalable PINN framework, bridging numerical iterative methods and deep learning, and opens avenues for rapid, mesh-free simulations in engineering and urban science. The approach is generic, easily integrated with standard optimizers and architectures, and can be extended to other PDEs and domains by choosing appropriate residual-smoothing operators and correction terms.

Abstract

Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN), a learning strategy that bridges modern physics-informed learning with numerical algorithms. Scale-PINN incorporates the iterative residual-correction principle, a cornerstone of numerical solvers, directly into the loss formulation, marking a paradigm shift in how PINN losses can be conceived and constructed. This integration enables Scale-PINN to achieve unprecedented convergence speed across PDE problems from different physics domain, including reducing training time on a challenging fluid-dynamics problem for state-of-the-art PINN from hours to sub-2 minutes while maintaining superior accuracy, and enabling application to representative problems in aerodynamics and urban science. By uniting the rigor of numerical methods with the flexibility of deep learning, Scale-PINN marks a significant leap toward the practical adoption of PINNs in science and engineering through scalable, physics-informed learning. Codes are available at https://github.com/chiuph/SCALE-PINN.

Figures (7)

• Figure 1: Scale-PINN schematic and result highlights. Scale-PINN includes a sequential correction term through application of the residual smoothing operator $\mathcal{P}_{\alpha} = (I - {\alpha^2} \nabla^2 )$ to the change in solution $\mathcal{F} \coloneqq f(\cdot;\boldsymbol{w}^{k})-f(\cdot;\boldsymbol{w}^{k-1})$ during iterative optimization. A convergence plot on the Navier-Stokes (N-S) example, lid‑driven cavity flow at $Re=3200$, shows competitive time‑to‑accuracy versus numerical solvers. Compared to other PINN methods, Scale-PINN solves the lid‑driven cavity flow to state-of-the-art accuracy with unprecedented speed, i.e., $\sim$90s for $Re=3200$ and $\sim$150s for $Re=7500$. Results for Kuramoto–Sivashinsky (K-S), Grey–Scott (G-S), Korteweg–De Vries (KdV), and Allen–Cahn (AC) equations demonstrate accuracy across diverse dynamics. Scale-PINN model architecture and training strategies are detailed in Method \ref{['sec:PINN-full']}.
• Figure 2: (a) Experimental analysis on lid‑driven cavity flow at $Re=400$ shows that the convergence of a vanilla PINN can be improved by increasing batch size (400$\rightarrow$4,000) and reducing learning rate (1e$^{\text{-3}}$$\rightarrow$1e$^{\text{-4}}$), albeit at a slower pace ($\sim$1800s). Scale-PINN requires substantially less training iterations to reach orders of magnitude higher accuracy ($\sim$90s), while using 1 order of magnitude smaller batch size and higher learning rate. Comparing their intermediate flow fields progressing from a few iterations to 50$k$-500$k$ iterations, and mid‑section profiles against the Ghia et al.bib:Ghia82 benchmark, Scale‑PINN attains accurate flow structures far earlier. (b) Scale-PINN can converge to an accurate solution even when the Reynolds number is increased to $Re=3200$, without the need to increase batch size and number of training iterations. A vanilla PINN struggles to solve the $Re=3200$ case, as it becomes trapped in incorrect flow patterns, indicating premature convergence.
• Figure 3: (a) Scale‑PINN solves the lid-driven cavity flow from $Re=400$ to $Re=20k$ with state‑of‑the‑art accuracy and efficient training, as shown in the summary table of error, training time, and other parameters, alongside the representative velocity fields and absolute error maps ($Re=400$ and $Re=20k$) to confirm that residuals remain small and largely confined to shear layers and vortex cores. For all simulated cases, their MSE consistently below 1e$^{\text{-4}}$, and their mid‑section $u$ and $v$-velocity profiles (colored lines) show excellent agreement with the classic benchmark results (marked points) for numerical solvers, i.e., Ghia bib:Ghia82 for up to $Re=10k$ and Erturk erturk2009discussions for $Re=20k$. (b) Scale-PINN establishes a sub-2 minutes training regime on lid-driven cavity flow ($Re=3200$), whereas recent PINN variants require hours to approach comparable accuracy. (c) Scale-PINN scales favorably with problem complexity ($Re$: 400$\rightarrow$3200), enabling the solution of more complex problems within a feasible time scale.
• Figure 4: Scale-PINN predictions, with streamlines overlaid on velocity magnitude contours, for flow past (a) a single-airfoil at $Re=1000$ and (b) staggered airfoils at $Re=500$ are compared to reference solutions obtained from CFD. Component-wise fields ($u$, $v$, $p$) and absolute error maps indicate good agreement between Scale-PINN and CFD across the domain, including wakes behind airfoil. Surface pressure‑coefficient ($C_{p}$) traces along the airfoil(s) closely match CFD and literature curves. Only the near-field region is shown for clarity, where the flow patterns around the airfoil(s) emerge; the actual computational domain extends well beyond the visualized region. Scale-PINN reaches accurate solutions within $\sim$180s of training, achieving near-field velocity relative errors of $1.7e^{\text{-2}}$ ($3.47e^{\text{-3}}$ full domain) for the single-airfoil case and $1.96e^{\text{-2}}$ ($4.79e^{\text{-3}}$ full domain) for the staggered airfoils case. These results validate the accuracy and efficiency of Scale-PINN in resolving canonical aerodynamic flow features at moderate Reynolds number.
• Figure 5: (a) Scale-PINN simulates the flow past square cylinders in open domain, where the predicted fields ($u$, $v$, $p$) accurately capture the wake structure and recirculation zones, with consistently low absolute errors against the reference solution obtained from CFD. Contours are shown in the near field for clarity; the actual computational domain extends further. Scale-PINN reaches accurate solutions within $\sim$285s of training, achieving near-field velocity relative errors of $9.21e^{\text{-3}}$ ($4.86e^{\text{-3}}$ full domain). (b) Scale-PINN simulates Rayleigh–Bénard convection at $Ra=100k$. The temperature contours, overlaid with velocity streamlines, show close agreement between predicted roll patterns and reference solutions obtained from CFD, across multiple time snapshots (1-50s). Scale-PINN achieves accurate solutions within $\sim$390s of training, with relative errors $1.99e^{\text{-2}}$ for velocity and $3.2e^{\text{-2}}$ for temperature. These cases validate the method’s robustness across bluff-body geometries in open domain and thermally driven, time-dependent convection.