Are Deep Learning Based Hybrid PDE Solvers Reliable? Why Training Paradigms and Update Strategies Matter

TL;DR

The paper addresses the reliability of deep learning–based hybrid iterative methods (DL-HIMs) for solving discretized PDEs, showing that outcomes depend sensitively on training paradigms and update strategies due to a spectral gap between classical smoothers and neural corrections. It analyzes fixed-point dynamics from both error and residual perspectives and compares static offline training with dynamic unrolling on two solvers: a DeepONet-based HINTS (nonlinear) and an FFT-based FNS (linear). A key contribution is the physics-aware Anderson acceleration (PA-AA), which minimizes the physical residual rather than the update magnitude, effectively breaking false fixed points and accelerating convergence in fewer iterations. The results demonstrate that reliability hinges on physically informed training and iteration design, not architectural expressiveness alone, with practical implications for deploying AI-based PDE solvers in scientific computing.

Abstract

Deep learning-based hybrid iterative methods (DL-HIMs) integrate classical numerical solvers with neural operators, utilizing their complementary spectral biases to accelerate convergence. Despite this promise, many DL-HIMs stagnate at false fixed points where neural updates vanish while the physical residual remains large, raising questions about reliability in scientific computing. In this paper, we provide evidence that performance is highly sensitive to training paradigms and update strategies, even when the neural architecture is fixed. Through a detailed study of a DeepONet-based hybrid iterative numerical transferable solver (HINTS) and an FFT-based Fourier neural solver (FNS), we show that significant physical residuals can persist when training objectives are not aligned with solver dynamics and problem physics. We further examine Anderson acceleration (AA) and demonstrate that its classical form is ill-suited for nonlinear neural operators. To overcome this, we introduce physics-aware Anderson acceleration (PA-AA), which minimizes the physical residual rather than the fixed-point update. Numerical experiments confirm that PA-AA restores reliable convergence in substantially fewer iterations. These findings provide a concrete answer to ongoing controversies surrounding AI-based PDE solvers: reliability hinges not only on architectures but on physically informed training and iteration design.

Figures (4)

• Figure 1: Update and Residual Norms of HINTS on the 1D Diffusion Equation with $N=801$
• Figure 2: Convergence Behavior of HINTS with DeepONet Models Trained in the Static Training Framework with Different Loss Functions.
• Figure 3: Convergence Behavior of FNS Statically Trained by Residual-Based and Error-Based Losses on Diffusion Equations ($N=61$).
• Figure 4: Convergence of DL-HIMs and Classic Numerical Methods with Different Update Strategies: Fixed Step Size (Fixed), Adaptive Step Size (Adaptive), Standard AA, and PA-AA.