Operator Learning at Machine Precision

TL;DR

This work introduces CHONKNORIS, a neural operator learning framework that achieves machine-precision accuracy by regressing the Cholesky factors of the regularized Gauss-Newton (Q) operator and unrolling a contractive Newton--Kantorovich (NK) update for PDEs and inverse problems. Building on CHONKNORIS, FONKNORIS extends to a foundation-model setting that aggregates multiple pre-trained experts to generalize across PDE families, enabling solution maps for unseen nonlinear PDEs such as Klein--Gordon and Sine--Gordon. The authors provide a rigorous inexact NK convergence analysis with Tikhonov regularization and demonstrate extensive numerical validation across forward nonlinear elliptic, Burgers, Darcy flow, and inverse problems (Calderón, inverse wave scattering, seismic imaging), all achieving near machine precision under appropriate iteration budgets. The results significantly elevate the accuracy and reliability of operator-learning approaches for PDEs, offering practical pathways for high-fidelity PDE emulation, solver acceleration, and robust PDE-constrained inference in scientific computing and digital-twin applications, with open-source code and data available for reproducibility.

Abstract

Neural operator learning methods have garnered significant attention in scientific computing for their ability to approximate infinite-dimensional operators. However, increasing their complexity often fails to substantially improve their accuracy, leaving them on par with much simpler approaches such as kernel methods and more traditional reduced-order models. In this article, we set out to address this shortcoming and introduce CHONKNORIS (Cholesky Newton--Kantorovich Neural Operator Residual Iterative System), an operator learning paradigm that can achieve machine precision. CHONKNORIS draws on numerical analysis: many nonlinear forward and inverse PDE problems are solvable by Newton-type methods. Rather than regressing the solution operator itself, our method regresses the Cholesky factors of the elliptic operator associated with Tikhonov-regularized Newton--Kantorovich updates. The resulting unrolled iteration yields a neural architecture whose machine-precision behavior follows from achieving a contractive map, requiring far lower accuracy than end-to-end approximation of the solution operator. We benchmark CHONKNORIS on a range of nonlinear forward and inverse problems, including a nonlinear elliptic equation, Burgers' equation, a nonlinear Darcy flow problem, the Calderón problem, an inverse wave scattering problem, and a problem from seismic imaging. We also present theoretical guarantees for the convergence of CHONKNORIS in terms of the accuracy of the emulated Cholesky factors. Additionally, we introduce a foundation model variant, FONKNORIS (Foundation Newton--Kantorovich Neural Operator Residual Iterative System), which aggregates multiple pre-trained CHONKNORIS experts for diverse PDEs to emulate the solution map of a novel nonlinear PDE. Our FONKNORIS model is able to accurately solve unseen nonlinear PDEs such as the Klein--Gordon and Sine--Gordon equations.

Figures (5)

• Figure 1: CHONKNORIS. (a) An initial guess $v_0$ for the true solution $v = \mathcal{G}(u)$ is iteratively refined by adding a correction term. (b) Each iteration consists of two steps: First, compute the correction term $\delta v = -(\widehat{\mathcal{R}}\widehat{\mathcal{R}}^* [{\tfrac{\delta \mathcal{F}}{\delta v}}]^*\mathcal{F})(u,v_k),$ where $\widehat{\mathcal{R}}$ is a learned surrogate for the Cholesky factors of $(\tfrac{\delta \mathcal{F}}{\delta v}\tfrac{\delta \mathcal{F}}{\delta v}^{T}+\lambda I)^{-1}$, $\mathcal{F}$ is the forward map with its Fréchet derivative $\tfrac{\delta \mathcal{F}}{\delta v}$, $v_k$ is the current approximation of the desired function $v=G(u)$, and $u$ is the input for which we seek the solution. Next, update the current approximation via $v_{k+1} = v_k + \delta v$.
• Figure 2: Forward problems. (a) Results for the nonlinear elliptic PDE problem. Quantiles of $10\%-90\%$ are shown across test realizations. Our CHONKNORIS method is able to achieve machine precision accuracy in around $10$ iterations. (b) Results for Burgers' equation. CHONKNORIS was able to achieve machine precision error in recovering the discretized solution which contained shocks. (c) Results for the Darcy flow PDE: (c1) shows that more challenging realizations require more CHONKNORIS iterations. (c4) shows a single realization of the random coefficient with the corresponding solution in (c2). (c3) shows the fixed forcing term.
• Figure 3: Quantiles of predictions of FONKNORIS for 100 realizations of initial conditions, external forces, and conductivities using a mixture of experts consisting of GPs for nonlinear elliptic, nonlinear darcy flow, and Burgers' equation and testing it for withheld Sine--Gordon and Klein--Gordon equations.
• Figure 4: Inverse Problems. (a) Results for the seismic imaging problem, showing the iterative solutions and the relative $L^2$ error across CHONKNORIS iteration. (b) For the seismic imaging problem, evolution of the adaptive regularization term in the Newton--Kantorovich iterations for different resolutions, and comparison of the relative $L^2$ error between the Newton--Kantorovich method and our CHONKNORIS method. (c) Results for the Calderón problem. (d) Results for the inverse wave scattering problem.
• Figure 5: Options for jointly adaptive relaxation and learning rate.

Theorems & Definitions (15)

• Remark 2.1: Reduction to parametric elliptic operator learning
• Remark 2.2: Interpolation between Gradient Descent and NK
• Theorem 4.2: Tikhonov--inexact Newton--Kantorovich
• Corollary 4.3: Convergence Rates
• Remark 4.4: Scheduling $\lambda_k$ to reach superlinear/quadratic convergence
• Remark 4.5: Variant: learning the full $B_\lambda(v)$ directly
• Example 4.6: Application to nonlinear elliptic PDE
• Theorem 4.7: Main convergence theorem with explicit constants
• Proposition B.1
• proof : Proof of \ref{['prop:lagrange_func_min']}