Neural Operator: Learning Maps Between Function Spaces
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar
TL;DR
This work introduces Neural Operator, a framework to learn mappings between infinite-dimensional function spaces, addressing the discretization sensitivity of conventional neural nets. By composing linear integral or spectral kernel operators with nonlinear activations, Neural Operators achieve discretization-invariance and universal approximation of continuous operators between Banach spaces. The authors develop four scalable parameterizations—Graph Neural Operators (GNO), Low-rank Operators (LNO), Multipole Graph Operators (MGNO), and Fourier Neural Operators (FNO)—and demonstrate state-of-the-art performance and speedups on PDE surrogate tasks for Poisson, Darcy, Burgers, and Navier–Stokes equations, including zero-shot super-resolution and Bayesian inverse problems. The paper also provides a formal approximation theory showing density of Neural Operators in appropriate operator spaces and situates Transformers and DeepONet within this continuum framework. Overall, Neural Operators offer mesh-agnostic, data-driven surrogates capable of rapid evaluation and robust generalization across discretizations, with broad implications for scientific computing and beyond.
Abstract
The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets. We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We formulate the neural operator as a composition of linear integral operators and nonlinear activation functions. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. The proposed neural operators are also discretization-invariant, i.e., they share the same model parameters among different discretization of the underlying function spaces. Furthermore, we introduce four classes of efficient parameterization, viz., graph neural operators, multi-pole graph neural operators, low-rank neural operators, and Fourier neural operators. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider standard PDEs such as the Burgers, Darcy subsurface flow, and the Navier-Stokes equations, and show that the proposed neural operators have superior performance compared to existing machine learning based methodologies, while being several orders of magnitude faster than conventional PDE solvers.