Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning

TL;DR

Kernel Neural Operators (KNOs) introduce a provably convergent operator-learning framework that stacks deep kernel-based integral operators with trainable, closed-form kernels. By decoupling the kernel from the numerical quadrature and allowing domain-specific quadrature rules, KNOs achieve geometric flexibility on irregular domains and use dimension-wise factorization to mitigate the curse of dimensionality on regular grids. The architecture combines a lifting operator, a sequence of integral operators with diagonal matrix-valued kernels (notably NS-GSM), nonlinear activations, and a final projection, enabling learnable operators from functions to functions with far fewer parameters than FFT-based or transformer-based rivals. Universal approximation guarantees are established for both infinite-dimensional and discretized formulations, and empirical results on seven benchmarks show competitive accuracy with substantially reduced parameter counts and memory footprints, including strong performance on irregular geometries and zero-shot super-resolution capabilities. These properties make KNOs attractive as memory-efficient surrogates for PDE operators and other function-to-function mappings in resource-constrained settings.

Abstract

This paper introduces the Kernel Neural Operator (KNO), a provably convergent operator-learning architecture that utilizes compositions of deep kernel-based integral operators for function-space approximation of operators (maps from functions to functions). The KNO decouples the choice of kernel from the numerical integration scheme (quadrature), thereby naturally allowing for operator learning with explicitly-chosen trainable kernels on irregular geometries. On irregular domains, this allows the KNO to utilize domain-specific quadrature rules. To help ameliorate the curse of dimensionality, we also leverage an efficient dimension-wise factorization algorithm on regular domains. More importantly, the ability to explicitly specify kernels also allows the use of highly expressive, non-stationary, neural anisotropic kernels whose parameters are computed by training neural networks. Numerical results demonstrate that on existing benchmarks the training and test accuracy of KNOs is comparable to or higher than popular operator learning techniques while typically using an order of magnitude fewer trainable parameters, with the more expressive kernels proving important to attaining high accuracy. KNOs thus facilitate low-memory, geometrically-flexible, deep operator learning, while retaining the implementation simplicity and transparency of traditional kernel methods from both scientific computing and machine learning.

Figures (5)

• Figure 1: Clustered quadrature points on $[0,1]^2$ (left) and a reference triangle (right).
• Figure 2: The 3D reaction-diffusion problem \ref{['sec:3d']}, where an input function is given (left), the true output function (center), and a prediction from the KNO (right).
• Figure 3: The quadrature rule used for for the 3D reaction-diffusion problem.
• Figure 4: Illustration of zero-shot super-resolution. The KNO was trained on the Darcy (PWC) dataset using a $29 \times 29$ grid (row a) and evaluated at a resolution of $211 \times 211$ (row b). The permeability field input (left), actual pressure field (middle), and predicted pressure field (right) are shown.
• Figure 5: An input pollutant concentration (left) and the corresponding ground truth and KNO predictions of output pollutant concentration (right) for the Beijing-Air problem.

Theorems & Definitions (10)

• Theorem 3.1
• proof
• Theorem 3.2
• proof
• proof
• Lemma B.1
• proof
• Lemma B.2
• proof
• proof