RRaPINNs: Residual Risk-Aware Physics Informed Neural Networks

TL;DR

RRaPINNs address a core limitation of PINNs: mean-residual optimization can hide localized, high-magnitude errors. By formulating training as a chance-constrained problem and adopting CVaR to penalize the tail of the PDE residual distribution, complemented by a Mean-Excess surrogate for stability, the method provides risk-sensitive control over worst-case errors. The approach is demonstrated across multiple PDEs, including smooth and discontinuous cases, showing improved tail behavior and preserved or enhanced mean accuracy relative to baselines. A two-phase training with adaptive tail budgeting offers a practical, drop-in extension for reliability-aware scientific ML in both smooth and discontinuous regimes.

Abstract

Physics-informed neural networks (PINNs) typically minimize average residuals, which can conceal large, localized errors. We propose Residual Risk-Aware Physics-Informed Neural Networks PINNs (RRaPINNs), a single-network framework that optimizes tail-focused objectives using Conditional Value-at-Risk (CVaR), we also introduced a Mean-Excess (ME) surrogate penalty to directly control worst-case PDE residuals. This casts PINN training as risk-sensitive optimization and links it to chance-constrained formulations. The method is effective and simple to implement. Across several partial differential equations (PDEs) such as Burgers, Heat, Korteweg-de-Vries, and Poisson (including a Poisson interface problem with a source jump at x=0.5) equations, RRaPINNs reduce tail residuals while maintaining or improving mean errors compared to vanilla PINNs, Residual-Based Attention and its variant using convolution weighting; the ME surrogate yields smoother optimization than a direct CVaR hinge. The chance constraint reliability level $α$ acts as a transparent knob trading bulk accuracy (lower $α$ ) for stricter tail control (higher $α$ ). We discuss the framework limitations, including memoryless sampling, global-only tail budgeting, and residual-centric risk, and outline remedies via persistent hard-point replay, local risk budgets, and multi-objective risk over BC/IC terms. RRaPINNs offer a practical path to reliability-aware scientific ML for both smooth and discontinuous PDEs.

Figures (28)

• Figure 1: Computational graph of a standard PINN. The neural network $u_\theta$ is evaluated at different sets of collocation points and processed through corresponding operators (PDE, BC, IC) to compute distinct loss terms. These terms are weighted (multiplied) and summed to form the final training objective $\widehat{\mathcal{L}}_{\text{PINN}}$.
• Figure 2: Reference solution for the 1D heat equation.
• Figure 3: Tail Plot for the 1D heat equation
• Figure 4: Exact Solution for the 2D Poisson equation.
• Figure 5: Tail Plot for the 2D Poisson equation

Theorems & Definitions (12)

• Remark 4.1
• Proposition 4.2: Proof
• Proposition 4.3: Proof
• Proposition 4.4: Proof
• Remark 1: Connection between CVaR and Average Top-$k$ Loss
• Proposition 2: Empirical RU minimizer and value
• proof
• Proposition 3: Empirical CVaR equals a fractional tail average (Proof of the proposition \ref{['prop_emp-cvar-fractional']})
• proof : Proof via the primal RU objective
• Proposition 4