Physics informed learning of orthogonal features with applications in solving partial differential equations

TL;DR

PD-OFM presents a physics-informed, orthogonality-regularized framework that learns operator-adaptive features for PDE solvers, addressing limitations of standard random feature methods. By jointly training a feature space and enforcing near-orthogonality, it yields bases that better capture the spectral structure of differential operators, reducing residuals by multiple orders of magnitude across Helmholtz, Poisson, wave, and Navier–Stokes problems. The method also demonstrates strong transferability, enabling features pretrained on one domain to efficiently solve others with the same operator, thereby lowering computational cost. The results highlight improved conditioning, higher effective rank, and lower projection error, offering a bridge between classical spectral methods and modern neural PDE solvers with practical implications for complex geometries and high-dimensional problems.

Abstract

The random feature method (RFM) constructs approximation spaces by initializing features from generic distributions, which provides universal approximation properties to solve general partial differential equations. However, such standard initializations lack awareness of the underlying physical laws and geometry, which limits approximation. In this work, we propose the Physics-Driven Orthogonal Feature Method (PD-OFM), a framework for constructing feature representations that are explicitly tailored to both the differential operator and the computational domain by pretraining features using physics-informed objectives together with orthogonality regularization. This pretraining strategy yields nearly orthogonal feature bases. We provide both theoretical and empirical evidence that physics-informed pretraining improves the approximation capability of the learned feature space. When employed to solve Helmholtz, Poisson, wave, and Navier-Stokes equations, the proposed method achieves residual errors 2-3 orders of magnitude lower than those of comparable methods. Furthermore, the orthogonality regularization improves transferability, enabling pretrained features to generalize effectively across different source terms and domain geometries for the same PDE.

Figures (14)

• Figure 1: The network has two parallel output branches: one predicts the solution $u(x; \theta)$, and the other outputs the feature space (green nodes).
• Figure 2: Approximation behavior of random features. The projection relies heavily on a small subset of features, indicating inefficient feature utilization.
• Figure 3: First 10 normalized basis functions learned by PD-OFM. The learned features remain well-distributed and non-degenerate, reflecting effective feature utilization.
• Figure 4: Approximation accuracy comparison between PD-OFM and the random feature method (RFM). PD-OFM exhibits consistently faster decay of the relative $L^2$ error as the number of features increases, achieving significantly higher accuracy at larger widths.
• Figure 5: Training dynamics comparison between PINN loss and least-squares solution error.

Theorems & Definitions (4)

• remark 1
• definition 1
• definition 2
• remark 2: Computational Complexity