Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hard-constraining Neumann boundary conditions in physics-informed neural networks via Fourier feature embeddings

Christopher Straub, Philipp Brendel, Vlad Medvedev, Andreas Rosskopf

TL;DR

The paper tackles the challenge of hard-constraining Neumann boundary conditions within physics-informed neural networks (PINNs) for diffusion-type problems, illustrated by a 1D model $\partial_t u = D\,\partial_x^2 u$ on $x\in[0,1]$ with Neumann data. It introduces a Fourier cosine feature embedding of the spatial input, where $\gamma(x)=(\cos(\pi x), \cos(\pi b_2 x), \ldots)$, so that the transformed solution $\tilde{u}(x,t)=u^{NN}(\gamma(x),t)$ satisfies vanishing Neumann conditions via the chain rule; non-vanishing BCs are handled by explicit additive terms, and the framework extends to general intervals and higher dimensions. The approach is evaluated against existing hard-constraining methods and vanilla PINNs on a forward diffusion problem, showing superior accuracy in high-frequency and multiscale settings, with favorable training-time profiles. The method is simple to implement, computationally efficient, and extensible to inverse problems and operator learning tasks, potentially broadening the applicability of PINNs to multiscale physical processes.

Abstract

We present a novel approach to hard-constrain Neumann boundary conditions in physics-informed neural networks (PINNs) using Fourier feature embeddings. Neumann boundary conditions are used to described critical processes in various application, yet they are more challenging to hard-constrain in PINNs than Dirichlet conditions. Our method employs specific Fourier feature embeddings to directly incorporate Neumann boundary conditions into the neural network's architecture instead of learning them. The embedding can be naturally extended by high frequency modes to better capture high frequency phenomena. We demonstrate the efficacy of our approach through experiments on a diffusion problem, for which our method outperforms existing hard-constraining methods and classical PINNs, particularly in multiscale and high frequency scenarios.

Hard-constraining Neumann boundary conditions in physics-informed neural networks via Fourier feature embeddings

TL;DR

The paper tackles the challenge of hard-constraining Neumann boundary conditions within physics-informed neural networks (PINNs) for diffusion-type problems, illustrated by a 1D model on with Neumann data. It introduces a Fourier cosine feature embedding of the spatial input, where , so that the transformed solution satisfies vanishing Neumann conditions via the chain rule; non-vanishing BCs are handled by explicit additive terms, and the framework extends to general intervals and higher dimensions. The approach is evaluated against existing hard-constraining methods and vanilla PINNs on a forward diffusion problem, showing superior accuracy in high-frequency and multiscale settings, with favorable training-time profiles. The method is simple to implement, computationally efficient, and extensible to inverse problems and operator learning tasks, potentially broadening the applicability of PINNs to multiscale physical processes.

Abstract

We present a novel approach to hard-constrain Neumann boundary conditions in physics-informed neural networks (PINNs) using Fourier feature embeddings. Neumann boundary conditions are used to described critical processes in various application, yet they are more challenging to hard-constrain in PINNs than Dirichlet conditions. Our method employs specific Fourier feature embeddings to directly incorporate Neumann boundary conditions into the neural network's architecture instead of learning them. The embedding can be naturally extended by high frequency modes to better capture high frequency phenomena. We demonstrate the efficacy of our approach through experiments on a diffusion problem, for which our method outperforms existing hard-constraining methods and classical PINNs, particularly in multiscale and high frequency scenarios.

Paper Structure

This paper contains 14 sections, 25 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Theoretical Background
  3. Physics-informed neural networks
  4. Existing hard-constraining techniques for Neumann boundary conditions
  5. Fourier feature embeddings
  6. Methodology
  7. Experiments
  8. Conclusion
  9. Theoretical background on the one-dimensional diffusion problem
  10. Methodology -- More general cases
  11. General intervals
  12. One-sided Neumann boundary condition
  13. Neumann boundary condition in higher dimensions
  14. More details on the experiments

Figures (3)

  • Figure 1: The proposed method to hard-constrain Neumann boundary conditions.
  • Figure 2: Relative improvements of the accuracies of PINNs with different hard-constraining (HC) and Fourier embedding strategies w.r.t. the reference PINN. All models have been trained for the same time. More details on the simulations are provided in Figure \ref{['fig:acc_bars_allit']} in appendix \ref{['sc:details_experiments']}.
  • Figure 3: Relative improvements of the accuracies of PINNs with different hard-constraining (HC) and Fourier embedding strategies w.r.t. a reference PINN in the settings described in Table \ref{['table:initials_heat']}. Different to Figure \ref{['fig:acc_bars']}, all models are trained for the same number of iterations ($10^6$ training iterations) here.