Unsupervised Physics-Informed Operator Learning through Multi-Stage Curriculum Training

TL;DR

This work reframes PDE solving as unsupervised physics-informed operator learning and introduces a principled multi-stage curriculum with optimizer resets to stabilize training. It couples a novel PhIS-FNO architecture, which merges Fourier layers with Hermite spline kernels, to enable smooth, boundary-aware residuals across periodic and non-periodic domains. Across Poisson, Burgers, Navier–Stokes, Kolmogorov flow, and Cylinder Wake benchmarks, the approach achieves convergence and accuracy comparable to supervised methods while maintaining resolution invariance, especially in non-periodic settings. Overall, the framework provides a robust, architecture-agnostic pathway for data-free, physics-consistent operator learning with strong practical potential for scalable PDE solvers.

Abstract

Solving partial differential equations remains a central challenge in scientific machine learning. Neural operators offer a promising route by learning mappings between function spaces and enabling resolution-independent inference, yet they typically require supervised data. Physics-informed neural networks address this limitation through unsupervised training with physical constraints but often suffer from unstable convergence and limited generalization capability. To overcome these issues, we introduce a multi-stage physics-informed training strategy that achieves convergence by progressively enforcing boundary conditions in the loss landscape and subsequently incorporating interior residuals. At each stage the optimizer is re-initialized, acting as a continuation mechanism that restores stability and prevents gradient stagnation. We further propose the Physics-Informed Spline Fourier Neural Operator (PhIS-FNO), combining Fourier layers with Hermite spline kernels for smooth residual evaluation. Across canonical benchmarks, PhIS-FNO attains a level of accuracy comparable to that of supervised learning, using labeled information only along a narrow boundary region, establishing staged, spline-based optimization as a robust paradigm for physics-informed operator learning.

Figures (15)

• Figure 1: Comparison among different training strategies on the Poisson benchmark. Each panel shows the magnitude of the PDE residual $|\Delta\psi + f|$ within the interior domain. The multi-stage approach (left) achieves the lowest residual and the most uniform convergence, whereas both the single-stage and the no-reset variants exhibit higher residuals and spatial oscillations, confirming the benefit of staged training and optimizer reinitialization.
• Figure 2: Validation loss comparison across epochs for the PhIS-FNO under different training strategies: multi-stage with reset (green), multi-stage without reset (purple), single-stage (blue) and supervised FNO (black). Sharp loss drops correspond to stage transitions in the multi-stage scheme.
• Figure 3: Comparison of training strategies for 2D Navier--Stokes and Kolmogorov flow. Each panel shows the validation $L_2$ loss versus training epochs across the full forecasting horizon $T$ for three architectures (PhIS-FNO, PINO, and UNet), two viscosity regimes $\nu=10^{-3}$ (top row) and $\nu=10^{-4}$ (middle row) and Kolmogorov flow (bottom row). The green curves correspond to the multi-stage (M-S) configuration with optimizer reset, the blue curves to multi-stage without reset (M-S nr), and the purple curves to the single-stage (S-S) setup with $\lambda_{\mathrm{res}}{=}1$. For numerical values of convergence see Table \ref{['tab:ns_kolm_all']}. Across all models and viscosities, the reset-based multi-stage strategy achieves the most stable convergence and lowest validation error, confirming its effectiveness in enforcing PDE consistency during training.
• Figure 4: Resolution-invariance of neural operator architectures. Relative error across spatial resolutions for different neural operator models: PINO (orange), PhIS--FNO (magenta), and UNet (cyan). Panels show results for (a) the 1D Burgers’ equation and (b,c) the 2D Navier--Stokes equations with viscosities $\nu=10^{-3}$ and $\nu=10^{-4}$. PINO and PhIS--FNO maintain stable accuracy across all grid sizes, indicating strong mesh invariance, whereas UNet exhibits a sharp degradation when evaluated away from the training resolution. These results highlight the ability of Fourier-based operator networks to generalize in a zero-shot manner to unseen spatial scales.
• Figure 5: Cylinder wake flow. Comparison between PhIS-FNO and PINO on the non-periodic cylinder wake benchmark. Panels (a,b) show the validation $L_2$ loss versus training epochs for both models under different training strategies: green = MS with optimizer reset, blue = MS nr, purple = SS, and black = fully supervised training. Panel (c) reports the boundary loss evolution during multi-stage training magenta: PhIS-FNO, orange: PINO). The reset-based multi-stage strategy yields faster and more stable convergence across both models, with PhIS-FNO achieving a markedly lower boundary loss ($1.44\times10^{-3}\,\pm\,4.59\times10^{-5}$) compared to PINO ($1.66\times10^{-2}\,\pm\,4.79\times10^{-3}$).