A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problems

TL;DR

The paper tackles the challenge of solving initial–boundary value problems (IBVPs) with neural operators that generalize to unseen initial and boundary conditions while training without numerical simulation data. It introduces PINTO, a physics-informed transformer operator that uses cross-attention-based iterative kernel integration to transform domain queries into boundary-aware representations, enabling fast, data-free generalization. The authors formalize the operator as $\mathcal{G}_\theta$, train it with a physics loss enforcing the PDE residuals and boundary constraints, and implement a three-stage architecture (lifting, cross-attention kernels, projection). Across five test cases including Advection, Burgers, and Navier–Stokes flows (Kovasznay, Beltrami, Lid-Driven Cavity), PINTO achieves substantially lower relative errors on unseen IB conditions compared with PI-DeepONet and even demonstrates temporal extrapolation beyond training times. This approach offers a scalable surrogate for fast PDE solving and data assimilation, with potential applications in design, control, and digital twins of fluid and transport phenomena.

Abstract

Initial boundary value problems arise commonly in applications with engineering and natural systems governed by nonlinear partial differential equations (PDEs). Operator learning is an emerging field for solving these equations by using a neural network to learn a map between infinite dimensional input and output function spaces. These neural operators are trained using a combination of data (observations or simulations) and PDE-residuals (physics-loss). A major drawback of existing neural approaches is the requirement to retrain with new initial/boundary conditions, and the necessity for a large amount of simulation data for training. We develop a physics-informed transformer neural operator (named PINTO) that efficiently generalizes to unseen initial and boundary conditions, trained in a simulation-free setting using only physics loss. The main innovation lies in our new iterative kernel integral operator units, implemented using cross-attention, to transform the PDE solution's domain points into an initial/boundary condition-aware representation vector, enabling efficient learning of the solution function for new scenarios. The PINTO architecture is applied to simulate the solutions of important equations used in engineering applications: advection, Burgers, and steady and unsteady Navier-Stokes equations (three flow scenarios). For these five test cases, we show that the relative errors during testing under challenging conditions of unseen initial/boundary conditions are only one-fifth to one-third of other leading physics informed operator learning methods. Moreover, our PINTO model is able to accurately solve the advection and Burgers equations at time steps that are not included in the training collocation points. The code is available at https://github.com/quest-lab-iisc/PINTO

Figures (14)

• Figure 1: Schematic of PINTO: Our neural operator has three stages (i) Query Point, Initial/Boundary Point and Initial/Boundary Value (lifting operators) (ii) Cross Attention Units (iterative kernel intergral operators) and (iii) output projection dense layers.
• Figure 2: Advection Equation: Initial conditions (first row), PINTO solutions (second row), corresponding PDEBENCH data (third row) and relative error of PINTO solution (fourth row) for seen (first two columns) and unseen (last two columns) initial conditions.
• Figure 3: Advection Equation: PINTO, PI-DeepONet and numerical solutions (PDEBENCH) for seen and unseen initial conditions at t=$0.01,\,0.5,\,2$. The first two columns are results for seen and the last columns are for the unseen initial conditions. A landmark A is shown in the solution for unseen initial conditions (ICs) to visualize how the wave is propagating in time.
• Figure 4: Learning curves during PINTO training for advection equation: a) Training (solid line) and validation (dashed line) loss curves for different learning rates. b) Training (solid line) and validation (dashed line) loss curves for different sequence lengths of initial conditions (input to the BPE/BVE) for a learning rate of $1e-5$.
• Figure 5: Burgers Equation: Initial conditions (first row), PINTO solutions (second row), corresponding numerical solution (third row) and relative error of PINTO solution (fourth row) for seen (first two columns) and unseen (last two columns) initial conditions.