Fourier PINNs: From Strong Boundary Conditions to Adaptive Fourier Bases

TL;DR

This work targets the spectral bias of Physics-Informed Neural Networks (PINNs) by combining a theoretical Fourier analysis of strong boundary condition PINNs with a practical Fourier PINN framework. The authors show, both analytically and numerically, that multiplying a neural network by a boundary function enhances learning of high-frequency components, and they introduce Fourier PINNs that augment the NN with trainable Fourier bases to flexibly capture dominant frequencies. An adaptive basis selection algorithm jointly optimizes neural and Fourier coefficients with basis pruning and L2 regularization, enabling robust identification of significant frequencies across problems. Empirically, Fourier PINNs outperform standard PINNs and several baselines on 1D/2D Poisson and Allen-Cahn problems, including high-frequency and multi-scale cases, while remaining BC- and domain-agnostic. The approach lays a path toward scalable, mesh-free PDE solvers with strong spectral fidelity, with future work focusing on tensor-decomposition strategies to extend to higher dimensions.

Abstract

Interest is rising in Physics-Informed Neural Networks (PINNs) as a mesh-free alternative to traditional numerical solvers for partial differential equations (PDEs). However, PINNs often struggle to learn high-frequency and multi-scale target solutions. To tackle this problem, we first study a strong Boundary Condition (BC) version of PINNs for Dirichlet BCs and observe a consistent decline in relative error compared to the standard PINNs. We then perform a theoretical analysis based on the Fourier transform and convolution theorem. We find that strong BC PINNs can better learn the amplitudes of high-frequency components of the target solutions. However, constructing the architecture for strong BC PINNs is difficult for many BCs and domain geometries. Enlightened by our theoretical analysis, we propose Fourier PINNs -- a simple, general, yet powerful method that augments PINNs with pre-specified, dense Fourier bases. Our proposed architecture likewise learns high-frequency components better but places no restrictions on the particular BCs or problem domains. We develop an adaptive learning and basis selection algorithm via alternating neural net basis optimization, Fourier and neural net basis coefficient estimation, and coefficient truncation. This scheme can flexibly identify the significant frequencies while weakening the nominal frequencies to better capture the target solution's power spectrum. We show the advantage of our approach through a set of systematic experiments.

Figures (21)

• Figure 1: This figure shows the polynomial distance function $\phi_{poly}(x)$ and the exponential distance function $\phi_{exp}(x)$ with different $\alpha$ values. The curves for $\phi_{exp}(x)$ correspond to different $\alpha$ parameters, demonstrating how the shape of $\phi_{exp}(x)$ "sharpens" around the domain boundaries as $\alpha$ increases.
• Figure 2: Relative $\ell_2$ error in predicting the solution $u(x)=\sin(k x)$ as a function of frequency $k$. The plots compare the performance of the standard PINN, the strong BC PINN with polynomial boundary function $\phi_{poly}$, the strong BC PINN with exponential boundary function $\phi_{exp}$, and FDM for (left) 1D Poisson and (right) 1D steady-state Allen-Cahn equations for NN with $2$ hidden layers (d-2) and $4$ hidden layers (d-4) each with $100$ neurons. The shaded regions represent the error variability.
• Figure 3: Frequency spectrum of the learned solution (left) and the convolution operation (right) in the strong BC PINN and standard PINN. The ground-truth solution is $\sin(k x)$ with $k=15$. The left graph shows frequency handling by standard and strong BC PINNs, while the right demonstrates how convolution reduces high-frequency noise.
• Figure 4: The frequency spectrum of the learned solutions (top) and absolute error of the Fourier coefficients (bottom) for the strong BC PINN and standard PINN compared to the ground truth. The ground-truth solution is $u(x) = \sin(2x) + \sin(16x)$.
• Figure 5: The frequency spectrum of the learned solutions (top) and absolute error of the Fourier coefficients (bottom) for the strong BC PINN and standard PINN compared to the ground truth. The ground-truth solution is $u(x) = e^{-0.5 x^2} \sin(16x)$.