NOWS: Neural Operator Warm Starts for Accelerating Iterative Solvers

TL;DR

This work introduces Neural Operator Warm Starts (NOWS), a hybrid strategy that harnesses learned solution operators to accelerate classical iterative solvers by producing high-quality initial guesses for Krylov methods such as conjugate gradient and GMRES.

Abstract

Partial differential equations (PDEs) underpin quantitative descriptions across the physical sciences and engineering, yet high-fidelity simulation remains a major computational bottleneck for many-query, real-time, and design tasks. Data-driven surrogates can be strikingly fast but are often unreliable when applied outside their training distribution. Here we introduce Neural Operator Warm Starts (NOWS), a hybrid strategy that harnesses learned solution operators to accelerate classical iterative solvers by producing high-quality initial guesses for Krylov methods such as conjugate gradient and GMRES. NOWS leaves existing discretizations and solver infrastructures intact, integrating seamlessly with finite-difference, finite-element, isogeometric analysis, finite volume method, etc. Across our benchmarks, the learned initialization consistently reduces iteration counts and end-to-end runtime, resulting in a reduction of the computational time of up to 90 %, while preserving the stability and convergence guarantees of the underlying numerical algorithms. By combining the rapid inference of neural operators with the rigor of traditional solvers, NOWS provides a practical and trustworthy approach to accelerate high-fidelity PDE simulations.

Figures (10)

• Figure 1: Workflow of the Neural Operator Warm Start (NOWS) framework. Schematic overview of the NOWS methodology integrating neural operators and numerical solvers. The left branch illustrates the data creation phase, where data are generated, if needed (not for PI versions), using traditional solvers. The middle part describes training phase, where a Neural Operator is trained through a combination of data, physics, and boundary losses. The architecture cycles through lifting, integral kernels (Fourier layers), activation, and projection steps until convergence. The right branch shows the inference phase, in which parametric PDE problems are discretized (e.g., using FEM, IGA, FVM). The trained Neural Operator provides an initial guess for a conventional iterative solver (e.g., CG, GMRES), which then refines the solution to full numerical accuracy. This hybrid workflow leverages the efficiency of learned models and the robustness of classical solvers to accelerate PDE solvers.
• Figure 1: Workflow of the Neural Operator Warm Start (NOWS) framework. Schematic overview of the NOWS methodology integrating neural operators and numerical solvers. The left branch illustrates the data creation phase, where data are generated, if needed (not for PI versions), using traditional solvers. The middle part describes training phase, where a Neural Operator is trained through a combination of data, physics, and boundary losses. The architecture cycles through lifting, integral kernels (Fourier layers), activation, and projection steps until convergence. The right branch shows the inference phase, in which parametric PDE problems are discretized (e.g., using FEM, IGA, FVM). The trained Neural Operator provides an initial guess for a conventional iterative solver (e.g., CG, GMRES), which then refines the solution to full numerical accuracy. This hybrid workflow leverages the efficiency of learned models and the robustness of classical solvers to accelerate PDE solvers.
• Figure 2: NOWS accelerates iterative solvers in various resolutions. (a) Overlaid residual-vs-iteration trajectories for the two methods on the 2000-sample test set, showing that NOWS reduces both the initial residual and the number of iterations to convergence. (b) Violin plots of wall-clock time distributions (per test instance) required to reach several relative residual tolerances, comparing the baseline and NOWS. (c) An example test case comparing the NOWS solution, the reference obtained with the direct solver. The NOWS removes the large-scale error, while the classical solver enforces exactness. Results are aggregated over 2000 unseen GRF realizations.
• Figure 2: NOWS accelerates iterative solvers in various resolutions. (a) Overlaid residual-vs-iteration trajectories for the two methods on the 2000-sample test set, showing that NOWS reduces both the initial residual and the number of iterations to convergence. (b) Violin plots of wall-clock time distributions (per test instance) required to reach several relative residual tolerances, comparing the baseline and NOWS. (c) An example test case comparing the NOWS solution, the reference obtained with the direct solver. The NOWS removes the large-scale error, while the classical solver enforces exactness. Results are aggregated over 2000 unseen GRF realizations.
• Figure 3: Impact of physics-informed training and neural-operator warm starts (NOWS) on Darcy flow simulations.(a) Test error as a function of network complexity, comparing data-only, physics-only, and hybrid training strategies. Physics-informed supervision yields the most robust generalization, with test error decreasing as the number of Fourier modes or network parameters increases, whereas data-only and hybrid approaches exhibit overfitting. Embedding the PDE directly in the loss produces smoother and more stable operator predictions that respect the underlying physics across resolutions and heterogeneous permeability contrasts. (b) Wall-clock runtime distributions for various preconditioners and relative tolerance levels, with and without NOWS initialization. Across all preconditioners, NOWS consistently reduces solver runtime, demonstrating measurable acceleration even for simple Jacobi preconditioning, with more pronounced improvements for ICC and ILU schemes. (c) Relative residuals versus iteration count for different preconditioners. NOWS systematically lowers the initial residual and reduces the number of iterations required to achieve the prescribed convergence tolerance, illustrating solver-agnostic acceleration and robustness across preconditioning strategies.