Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latent Neural Operator for Solving Forward and Inverse PDE Problems

Tian Wang, Chuang Wang

TL;DR

This work introduces Latent Neural Operator (LNO), a Transformer-based framework that solves forward and inverse PDE problems by learning a latent space representation through Physics-Cross-Attention (PhCA). By encoding inputs into a learnable latent space, applying a latent-space operator, and decoding back to the physical space, LNO achieves high accuracy with substantially reduced memory and training time. The approach enables interpolation and extrapolation at unseen query locations, improving performance on inverse problems, and demonstrates state-of-the-art results on multiple forward benchmarks with strong generalization capabilities. Overall, LNO offers a scalable, efficient, and flexible alternative to traditional PDE solvers and existing neural operators, with practical impact for data-driven scientific computing.

Abstract

Neural operators effectively solve PDE problems from data without knowing the explicit equations, which learn the map from the input sequences of observed samples to the predicted values. Most existing works build the model in the original geometric space, leading to high computational costs when the number of sample points is large. We present the Latent Neural Operator (LNO) solving PDEs in the latent space. In particular, we first propose Physics-Cross-Attention (PhCA) transforming representation from the geometric space to the latent space, then learn the operator in the latent space, and finally recover the real-world geometric space via the inverse PhCA map. Our model retains flexibility that can decode values in any position not limited to locations defined in the training set, and therefore can naturally perform interpolation and extrapolation tasks particularly useful for inverse problems. Moreover, the proposed LNO improves both prediction accuracy and computational efficiency. Experiments show that LNO reduces the GPU memory by 50%, speeds up training 1.8 times, and reaches state-of-the-art accuracy on four out of six benchmarks for forward problems and a benchmark for inverse problem. Code is available at https://github.com/L-I-M-I-T/LatentNeuralOperator.

Latent Neural Operator for Solving Forward and Inverse PDE Problems

TL;DR

This work introduces Latent Neural Operator (LNO), a Transformer-based framework that solves forward and inverse PDE problems by learning a latent space representation through Physics-Cross-Attention (PhCA). By encoding inputs into a learnable latent space, applying a latent-space operator, and decoding back to the physical space, LNO achieves high accuracy with substantially reduced memory and training time. The approach enables interpolation and extrapolation at unseen query locations, improving performance on inverse problems, and demonstrates state-of-the-art results on multiple forward benchmarks with strong generalization capabilities. Overall, LNO offers a scalable, efficient, and flexible alternative to traditional PDE solvers and existing neural operators, with practical impact for data-driven scientific computing.

Abstract

Neural operators effectively solve PDE problems from data without knowing the explicit equations, which learn the map from the input sequences of observed samples to the predicted values. Most existing works build the model in the original geometric space, leading to high computational costs when the number of sample points is large. We present the Latent Neural Operator (LNO) solving PDEs in the latent space. In particular, we first propose Physics-Cross-Attention (PhCA) transforming representation from the geometric space to the latent space, then learn the operator in the latent space, and finally recover the real-world geometric space via the inverse PhCA map. Our model retains flexibility that can decode values in any position not limited to locations defined in the training set, and therefore can naturally perform interpolation and extrapolation tasks particularly useful for inverse problems. Moreover, the proposed LNO improves both prediction accuracy and computational efficiency. Experiments show that LNO reduces the GPU memory by 50%, speeds up training 1.8 times, and reaches state-of-the-art accuracy on four out of six benchmarks for forward problems and a benchmark for inverse problem. Code is available at https://github.com/L-I-M-I-T/LatentNeuralOperator.
Paper Structure (29 sections, 8 equations, 8 figures, 10 tables)

This paper contains 29 sections, 8 equations, 8 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Physics-Informed Machine Learning
  4. Operator Learning
  5. Inverse Problem
  6. Method
  7. Problem Setup
  8. Latent Neural Operator
  9. Overview
  10. Embedding inputs
  11. Encoding to the latent space
  12. Learning operator in the latent space
  13. Decoding to the geometric space
  14. Physics-Cross-Attention
  15. Experiments
  16. ...and 14 more sections

Figures (8)

  • Figure 1: The overall architecture of Latent Neural Operator.
  • Figure 2: The working mechanism of Physics-Cross-Attention in encoder and decoder respectively.
  • Figure 3: Sampling numbers in latent space v.s. Accuracy. (a) Forward problems on Airfoil, Elasticity, Plasticity and Pipe. (b) Forward problems on Darcy and NS2d. (c) Inverse Problem.
  • Figure 4: The influence of model depth on the performance of LNO across six forward problem benchmarks.
  • Figure 5: The influence of model width on the performance of LNO across six forward problem benchmarks.
  • ...and 3 more figures