Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FieldFormer: Physics-Informed Transformers for Spatio-Temporal Field Reconstruction from Sparse Sensors

Ankit Bhardwaj, Ananth Balashankar, Lakshminarayanan Subramanian

TL;DR

FieldFormer tackles mesh-free spatio-temporal field reconstruction from sparse, noisy observations by introducing a local, anisotropy-aware transformer that uses a velocity-scaled neighborhood metric and a precomputed offset table, coupled with autograd-based PDE residual penalties. It establishes a finite-order local operator universal-approximation property and demonstrates state-of-the-art accuracy across heat, shallow-water, and pollution benchmarks, achieving RMSEs near $10^{-2}$ under sparse data regimes. The approach blends data-driven flexibility with physics-based regularization to yield accurate, scalable reconstructions in environmental and urban sensing contexts where data are scarce and irregular.

Abstract

Spatio-temporal sensor data is often sparse, noisy, and irregular, and existing interpolation or learning methods struggle here because they either ignore governing PDEs or do not scale. We introduce FieldFormer, a transformer-based framework for mesh-free spatio-temporal field reconstruction that combines data-driven flexibility with physics-based structure. For each query, FieldFormer gathers a local neighborhood using a learnable velocity-scaled distance metric, enabling anisotropic adaptation to different propagation regimes. Neighborhoods are built efficiently via per-batch offset recomputation, and refined in an expectation-maximization style as the velocity scales evolve. Predictions are made by a local transformer encoder, and physics consistency is enforced through autograd-based PDE residuals and boundary-specific penalties. Across three benchmarks--a scalar anisotropic heat equation, a vector-valued shallow-water system, and a realistic advection-diffusion pollution simulation--FieldFormer consistently outperforms strong baselines by more than 40%. Our results demonstrate that FieldFormer enables accurate (RMSE$<10^{-2}$), efficient, and physically consistent field reconstruction from sparse (0.4%-2%) and noisy(10%) data.

FieldFormer: Physics-Informed Transformers for Spatio-Temporal Field Reconstruction from Sparse Sensors

TL;DR

FieldFormer tackles mesh-free spatio-temporal field reconstruction from sparse, noisy observations by introducing a local, anisotropy-aware transformer that uses a velocity-scaled neighborhood metric and a precomputed offset table, coupled with autograd-based PDE residual penalties. It establishes a finite-order local operator universal-approximation property and demonstrates state-of-the-art accuracy across heat, shallow-water, and pollution benchmarks, achieving RMSEs near under sparse data regimes. The approach blends data-driven flexibility with physics-based regularization to yield accurate, scalable reconstructions in environmental and urban sensing contexts where data are scarce and irregular.

Abstract

Spatio-temporal sensor data is often sparse, noisy, and irregular, and existing interpolation or learning methods struggle here because they either ignore governing PDEs or do not scale. We introduce FieldFormer, a transformer-based framework for mesh-free spatio-temporal field reconstruction that combines data-driven flexibility with physics-based structure. For each query, FieldFormer gathers a local neighborhood using a learnable velocity-scaled distance metric, enabling anisotropic adaptation to different propagation regimes. Neighborhoods are built efficiently via per-batch offset recomputation, and refined in an expectation-maximization style as the velocity scales evolve. Predictions are made by a local transformer encoder, and physics consistency is enforced through autograd-based PDE residuals and boundary-specific penalties. Across three benchmarks--a scalar anisotropic heat equation, a vector-valued shallow-water system, and a realistic advection-diffusion pollution simulation--FieldFormer consistently outperforms strong baselines by more than 40%. Our results demonstrate that FieldFormer enables accurate (RMSE), efficient, and physically consistent field reconstruction from sparse (0.4%-2%) and noisy(10%) data.

Paper Structure

This paper contains 12 sections, 2 theorems, 31 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. FieldFormer Architecture
  5. Physics-Informed Loss via Autograd
  6. Inference and Efficiency
  7. Formal Analysis
  8. Evaluation
  9. Baselines and Metrics
  10. Evaluation Settings and Results
  11. Discussion and Conclusion
  12. Acknowledgements

Key Result

Theorem 3.2

Let $u:\mathbb{R}^{d}\times\mathbb{R}\to\mathbb{R}^q$ solve a parabolic or hyperbolic PDE with a finite-order, local operator. Assume a consistent explicit finite-difference scheme with compact stencil $\mathcal{S}\subset\{-s_{\max},\ldots,s_{\max}\}^d\times\{0,\ldots,k\}$ and a CFL-admissible time step $\Delta t$. Then for any $\varepsilon>0$ and compact $K\subset\mathbb{R}^{d+1}$, ther

Figures (1)

  • Figure 1: Overview of FieldFormer-Autograd architecture for PDE-conformant field imputation.

Theorems & Definitions (7)

  • Definition 3.1: Finite-order local differential operator
  • Theorem 3.2: Universal Approximation for Parabolic and Hyperbolic PDEs
  • proof : Proof sketch
  • Remark 3.3: Implication: Suitability for Modeling Local Processes
  • Theorem 3.7: Average-Case Lower Bound Under Longitudinal Sampling
  • proof : Proof Sketch
  • Remark 3.8: Implication: Effect of Stencil Size on Miss-Coverage Error