SetPINNs: Set-based Physics-informed Neural Networks

TL;DR

SetPINNs address a core limitation of pointwise PINNs by incorporating local domain dependencies through FEM-inspired domain partitioning and set-based attention. They prove that element-aware sampling yields unbiased, lower-variance estimates of the PDE residual energy $I(\theta)=\int_\Omega \|\mathcal{O}_\Omega(u_\theta)(x)\|^2 dx$ and its gradients, improving domain coverage and training stability. Empirically, SetPINNs outperform strong baselines across white-box PDEs and grey-box chemical-process tasks, with substantial gains in accuracy and robustness and favorable wall-clock performance. This framework advances scalable, physics-consistent learning for complex PDEs, especially in high-dimensional or irregular domains, by jointly leveraging locality and permutation-equivariant set processing.

Abstract

Physics-Informed Neural Networks (PINNs) solve partial differential equations using deep learning. However, conventional PINNs perform pointwise predictions that neglect dependencies within a domain, which may result in suboptimal solutions. We introduce SetPINNs, a framework that effectively captures local dependencies. With a finite element-inspired sampling scheme, we partition the domain into sets to model local dependencies while simultaneously enforcing physical laws. We provide a rigorous theoretical analysis showing that SetPINNs yield unbiased, lower-variance estimates of residual energy and its gradients, ensuring improved domain coverage and reduced residual error. Extensive experiments on synthetic and real-world tasks show improved accuracy, efficiency, and robustness.

Figures (9)

• Figure 1: Comparison between (a) standard Physics-Informed Neural Networks (PINNs) and (b) the proposed Set-based PINNs (SetPINNs). Standard PINNs predict $u_\theta(x_i)$ independently at sparse, uniformly sampled points $x_i \in \Omega$, often underrepresenting parts of the domain $\Omega$ and lacking local context. SetPINNs partition the domain into local elements $E_k$, jointly conditioning predictions on sampled sets and capturing local structure. This promotes better coverage, stronger inductive bias, and improved physical consistency.
• Figure 2: Overview of the SetPINNs framework. On the left, we exemplify the set generation process for diverse physical domains. Sets are formed from neighboring points in the domain. Each set is then processed through a Mixer Network, Set Processor, and PDE Probe to predict the solution for each element in the set. The parameters of SetPINNs are learned using a localized energy residual.
• Figure 3: Setup and performance comparison for a clamped plate with a localized heavy load. (a) Illustration of the plate and load placement, representing a common structural engineering scenario. (b) Comparison of SetPINNs and baseline methods against the true solution, highlighting SetPINNs’ ability to capture high-stress regions due to element-aware sampling and a set-based architecture.
• Figure 4: Loss landscape of SetPINNs compared to baselines. Baseline models exhibit sharp cones in their loss landscape, whereas SetPINNs demonstrate a significantly smoother loss surface. This indicates that SetPINNs optimize more stably, reducing sensitivity to failure modes.
• Figure 5: SetPINN design and sampling analysis. Left: SetPINN error and Coverage Ratio (CR) vs. element size. All tested SetPINN configurations outperform the standard PINN baseline in both error and CR. Smaller SetPINN elements further improves error and CR by reducing intra-set distances. Right: Relative rRMSE improvement (%) of vanilla PINNs with advanced sampling (EAS, LHS, RAD) over uniform sampling. Proposed EAS typically provides the largest benefits.

Theorems & Definitions (11)

• Definition 3.1: Domain Partitioning
• Definition 3.2: Element-Aware Sampling (EAS)
• Definition 4.1: Coverage Ratio
• Lemma 4.2: Unbiased Estimation via Sampling
• Theorem 4.3: Variance Reduction with EAS
• Theorem 4.4: Gradient Variance Reduction with EAS
• proof : Proof of Lemma \ref{['lem:unbiased_estimation']}
• proof : Proof of Theorem \ref{['thm:variance_reduction']}
• Corollary A.1: Improved Reliability of Coverage Metric
• proof : Proof of Corollary \ref{['cor:reliability']}