Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Neural Operators from Partial Observations via Latent Autoregressive Modeling

Jingren Hou, Hong Wang, Pengyu Xu, Chang Gao, Huafeng Liu, Liping Jing

TL;DR

This work tackles the challenge of learning neural operators when observations are incomplete by identifying supervision gaps and input–output misalignment as core obstacles. It introduces Mask-to-Predict (MPT) to generate artificial supervision and Latent Autoregressive Neural Operator (LANO) with a Physics-Aware Latent Propagator (PhLP) to progressively reconstruct unobserved regions from boundary information in latent space. The approach is validated on the POBench-PDE benchmark across three PDE tasks, achieving 18%–69% relative L2 error reductions and demonstrating strong generalization under substantial missingness, including real-world climate data. These results bridge the gap between idealized laboratory setups and real-world PDE solving under data scarcity, enabling more robust, scalable, and practical neural operators for scientific computing.

Abstract

Real-world scientific applications frequently encounter incomplete observational data due to sensor limitations, geographic constraints, or measurement costs. Although neural operators significantly advanced PDE solving in terms of computational efficiency and accuracy, their underlying assumption of fully-observed spatial inputs severely restricts applicability in real-world applications. We introduce the first systematic framework for learning neural operators from partial observation. We identify and formalize two fundamental obstacles: (i) the supervision gap in unobserved regions that prevents effective learning of physical correlations, and (ii) the dynamic spatial mismatch between incomplete inputs and complete solution fields. Specifically, our proposed Latent Autoregressive Neural Operator(LANO) introduces two novel components designed explicitly to address the core difficulties of partial observations: (i) a mask-to-predict training strategy that creates artificial supervision by strategically masking observed regions, and (ii) a Physics-Aware Latent Propagator that reconstructs solutions through boundary-first autoregressive generation in latent space. Additionally, we develop POBench-PDE, a dedicated and comprehensive benchmark designed specifically for evaluating neural operators under partial observation conditions across three PDE-governed tasks. LANO achieves state-of-the-art performance with 18--69$\%$ relative L2 error reduction across all benchmarks under patch-wise missingness with less than 50$\%$ missing rate, including real-world climate prediction. Our approach effectively addresses practical scenarios involving up to 75$\%$ missing rate, to some extent bridging the existing gap between idealized research settings and the complexities of real-world scientific computing.

Learning Neural Operators from Partial Observations via Latent Autoregressive Modeling

TL;DR

This work tackles the challenge of learning neural operators when observations are incomplete by identifying supervision gaps and input–output misalignment as core obstacles. It introduces Mask-to-Predict (MPT) to generate artificial supervision and Latent Autoregressive Neural Operator (LANO) with a Physics-Aware Latent Propagator (PhLP) to progressively reconstruct unobserved regions from boundary information in latent space. The approach is validated on the POBench-PDE benchmark across three PDE tasks, achieving 18%–69% relative L2 error reductions and demonstrating strong generalization under substantial missingness, including real-world climate data. These results bridge the gap between idealized laboratory setups and real-world PDE solving under data scarcity, enabling more robust, scalable, and practical neural operators for scientific computing.

Abstract

Real-world scientific applications frequently encounter incomplete observational data due to sensor limitations, geographic constraints, or measurement costs. Although neural operators significantly advanced PDE solving in terms of computational efficiency and accuracy, their underlying assumption of fully-observed spatial inputs severely restricts applicability in real-world applications. We introduce the first systematic framework for learning neural operators from partial observation. We identify and formalize two fundamental obstacles: (i) the supervision gap in unobserved regions that prevents effective learning of physical correlations, and (ii) the dynamic spatial mismatch between incomplete inputs and complete solution fields. Specifically, our proposed Latent Autoregressive Neural Operator(LANO) introduces two novel components designed explicitly to address the core difficulties of partial observations: (i) a mask-to-predict training strategy that creates artificial supervision by strategically masking observed regions, and (ii) a Physics-Aware Latent Propagator that reconstructs solutions through boundary-first autoregressive generation in latent space. Additionally, we develop POBench-PDE, a dedicated and comprehensive benchmark designed specifically for evaluating neural operators under partial observation conditions across three PDE-governed tasks. LANO achieves state-of-the-art performance with 18--69 relative L2 error reduction across all benchmarks under patch-wise missingness with less than 50 missing rate, including real-world climate prediction. Our approach effectively addresses practical scenarios involving up to 75 missing rate, to some extent bridging the existing gap between idealized research settings and the complexities of real-world scientific computing.
Paper Structure (40 sections, 1 theorem, 7 equations, 5 figures, 3 tables)

This paper contains 40 sections, 1 theorem, 7 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Our approach
  3. Related Work
  4. Neural Operator Learning.
  5. Training under Sparse Observation.
  6. Proposed Method
  7. Overview of LANO
  8. Problem setup.
  9. Objective.
  10. Mask-to-predict training strategy
  11. Boundary-first autoregressive framework
  12. Model Architecture
  13. Overview.
  14. Architecture.
  15. Temporal Aggregation Layer.
  16. ...and 25 more sections

Key Result

Theorem 3.1

Let $\mathcal{D} \subset \mathbb{R}^d$ be the computational domain and $\mathcal{D}_{o}^l \subseteq \mathcal{D}$ the observed region at layer $l$. Given features $\mathbf{Y}^{l-1}: \mathcal{D} \to \mathbb{R}^C$, PhLP is equivalent to applying the learnable integral operator where the kernel $\kappa^l(\mathbf{x}^*, \boldsymbol{\xi}) \approx \sum_{h=1}^H \sum_{k=1}^L \phi^l_{hk}(\mathbf{x}^*) \psi^

Figures (5)

  • Figure 1: An overview of neural operator learning under partially observed PDE datasets. Once trained, the model is expected to infer solutions from unseen, partially observed inputs. However, two challenges arise in this setting, as summarized on the right.
  • Figure 2: Overview of the proposed LANO model architecture. We begin by sampling trajectories from partial observations. The model is optimized using the mask-to-predict training strategy (MPT), where artificially masked past frames are used to predict the next frame. To support this learning objective, we design a novel model architecture incorporating a physics-aware latent propagator (PhLP), which implements a boundary-first autoregressive framework.
  • Figure 3: (a) Visualization of boundary-first latent propagation in PhLP. Each row shows different masked trajectory variants of the same historical sequence. From left to right: features evolve from shallow to deep layers, demonstrating progressive reconstruction from partial observations to coherent physical representations. See Appendix D.2 for more visualizations. (b) Effectiveness of Mask-to-Predict Training strategy (MPT) on Diffusion-Reaction dataset. Heatmaps show relative L2 error across different data sizes and missing rates under patch-wise missingness. Left: without MPT, Right: with MPT. Lower is better.
  • Figure 4: Performance under varying patch sizes on Navier-Stokes (25% patch-wise missingness). LANO achieves consistent improvements of 18.5%-25.0% over LNO wang2024latent. Remarkably, our experiments demonstrate that even under 25% patch-wise missingness, LANO achieves performance comparable to or exceeding that of FNO li2020fourier, one of the foundational works in neural operator learning.
  • Figure 5: Evaluation of the model scalability in terms of data size and parameter size. Default setting: 1000 cases, 8 layers.

Theorems & Definitions (2)

  • Theorem 3.1: PhLP as an Integral Operator with Self-Update (Simplified)
  • proof : Proof Sketch