GradINN: Gradient Informed Neural Network

TL;DR

The advantages of GradINNs, particularly in low-data regimes, are demonstrated on diverse problems spanning non time-dependent systems (Friedman function, Stokes Flow) and time-dependent systems (Lotka-Volterra, Burger's equation).

Abstract

We propose Gradient Informed Neural Networks (GradINNs), a methodology inspired by Physics Informed Neural Networks (PINNs) that can be used to efficiently approximate a wide range of physical systems for which the underlying governing equations are completely unknown or cannot be defined, a condition that is often met in complex engineering problems. GradINNs leverage prior beliefs about a system's gradient to constrain the predicted function's gradient across all input dimensions. This is achieved using two neural networks: one modeling the target function and an auxiliary network expressing prior beliefs, e.g., smoothness. A customized loss function enables training the first network while enforcing gradient constraints derived from the auxiliary network. We demonstrate the advantages of GradINNs, particularly in low-data regimes, on diverse problems spanning non time-dependent systems (Friedman function, Stokes Flow) and time-dependent systems (Lotka-Volterra, Burger's equation). Experimental results showcase strong performance compared to standard neural networks and PINN-like approaches across all tested scenarios.

Figures (9)

• Figure 1: Ground truth (black dashed lines) and predicted solution (left) and gradient (right) both at initialization (gray lines) and after training. Predictions for gradinn are obtained with $m=100$ collocation points uniformly distributed $[-2,3]$.
• Figure 2: friedman. Predicted output $u$ vs $u$ for each $\boldsymbol{x}_n \in \mathcal{D}_{\textsc{test}\xspace}$Top: s-nn with $\textsc{rmse}\xspace_U = 0.51$. Bottom: gradinn with $\textsc{rmse}\xspace_U = 0.04$.
• Figure 3: stokes. Top: $\textsc{rmse}\xspace_U$ over training epochs when constraining $\nabla_{\boldsymbol{x}}u$ (black line) and both the $\nabla_{\boldsymbol{x}}u$ and $\nabla^2_{\boldsymbol{x}}u$ (red line). Bottom: predicted solution when constraining $\nabla_{\boldsymbol{x}}u$ and $\nabla^2_{\boldsymbol{x}}u$ ($n_{u}=350$).
• Figure 4: stokes. Predicted solution and ground truth values (first column) for different $n_{u}$ values (white dotted lines). Top:$n_{u} = 350$. Middle:$n_{u} = 550$. Bottom:$n_{u} = 750$.
• Figure 5: lv. Top: Predicted solution. Bottom: Predicted gradient.