Enhancing Physics-Informed Neural Networks with Domain-aware Fourier Features: Towards Improved Performance and Interpretable Results

TL;DR

A novel modeling approach is proposed, which relies on the use of Domain-aware Fourier Features (DaFFs) for the positional encoding of the input space, demonstrating that DaFFs not only enhance PINNs' accuracy and efficiency but also improve interpretability.

Abstract

Physics-Informed Neural Networks (PINNs) incorporate physics into neural networks by embedding partial differential equations (PDEs) into their loss function. Despite their success in learning the underlying physics, PINN models remain difficult to train and interpret. In this work, a novel modeling approach is proposed, which relies on the use of Domain-aware Fourier Features (DaFFs) for the positional encoding of the input space. These features encapsulate all the domain-specific characteristics, such as the geometry and boundary conditions, and unlike Random Fourier Features (RFFs), eliminate the need for explicit boundary condition loss terms and loss balancing schemes, while simplifying the optimization process and reducing the computational cost associated with training. We further develop an LRP-based explainability framework tailored to PINNs, enabling the extraction of relevance attribution scores for the input space. It is demonstrated that PINN-DaFFs achieve orders-of-magnitude lower errors and allow faster convergence compared to vanilla PINNs and RFFs-based PINNs. Furthermore, LRP analysis reveals that the proposed leads to more physically consistent feature attributions, while PINN-RFFs and vanilla PINNs display more scattered and less physics-relevant patterns. These results demonstrate that DaFFs not only enhance PINNs' accuracy and efficiency but also improve interpretability, laying the ground for more robust and informative physics-informed learning.

Figures (10)

• Figure 1: Schematic representation of the proposed modeling approach.
• Figure 2: Learning phase for the Kirchhoff problem; the left column depicts the evolution of the training and validation losses over epochs; the PDE loss denoted by $L_{\mathrm{r}}$ and the boundary condition loss terms denoted by $L_{\mathrm{b}_1}$ and $L_{\mathrm{b}_2}$; the right column presents the weights assigned to the loss terms by the loss balancing strategy during training.
• Figure 3: Learning phase for the Helmholtz problem; the left column depicts the evolution of the training and validation losses over epochs; the PDE loss is denoted by $L_{\mathrm{r}}$ and the loss terms of the boundary condition loss terms are denoted by $L_{\mathrm{b}_1}$, $L_{\mathrm{b}_2}$, $L_{\mathrm{b}_3}$ and $L_{\mathrm{b}_4}$; the right column presents the weights assigned to the loss terms by the loss balancing strategy during training
• Figure 4: Solution to the Kirchhoff and Helmholtz numerical examples. Given the value distribution, the spectral components for Kirchhoff should approximate $sin(b*x)$ and $cos(b*x)$ where $x \in \Omega, b \approx \pi/2$, while for Helmholtz they are dependent on the coordinate, with $b \approx \pi$ for the $y$ component and $b \approx 4\pi$ for the $x$ component.
• Figure 5: Contribution of the $x$ and $y$ inputs to the prediction of the best vanilla PINN across the whole domain of the problems computed with LRP. On the left, $R$ is computed by LRP, where positive values indicate a stronger effect from $x$ and negative values indicate a higher impact from $y$. On the right, the contributions $R'$ of the PINN are filtered with an arbitrary threshold of 10, so that $if \quad |R(x,y)| > 10, \quad then |R(x,y)| = 10, \quad \forall (x,y) \in \Omega$.