Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Numerical Robustness of PINNs for Multiscale Transport Equations

Alexander Jesser, Kai Krycki, Ryan G. McClarren, Martin Frank

TL;DR

This work analyzes the numerical robustness of Physics Informed Neural Networks (PINNs) for multiscale transport equations, focusing on the diffusion limit in slab-geometry neutron transport. By drawing an analogy to Least-Squares Finite Elements (LSFE), the authors show that vanilla PINNs with ReLU activations fail to reproduce the correct diffusion limit as the mean free path becomes small, and they introduce a diffusive scaling (S = P + \tau(I - P)) to remedy this. Theoretical results, complemented by numerical experiments using PN angular discretization, demonstrate that the scaled PINN (and LSFE) solutions converge toward the diffusion solution across test cases, including interfaces and non-ReLU activations like tanh. The findings highlight the importance of asymptotic-preserving scalings for multiscale PDEs and suggest directions for formal convergence proofs and extensions to other activations and scalings.

Abstract

We investigate the numerical solution of multiscale transport equations using Physics Informed Neural Networks (PINNs) with ReLU activation functions. Therefore, we study the analogy between PINNs and Least-Squares Finite Elements (LSFE) which lies in the shared approach to reformulate the PDE solution as a minimization of a quadratic functional. We prove that in the diffusive regime, the correct limit is not reached, in agreement with known results for first-order LSFE. A diffusive scaling is introduced that can be applied to overcome this, again in full agreement with theoretical results for LSFE. We provide numerical results in the case of slab geometry that support our theoretical findings.

Numerical Robustness of PINNs for Multiscale Transport Equations

TL;DR

This work analyzes the numerical robustness of Physics Informed Neural Networks (PINNs) for multiscale transport equations, focusing on the diffusion limit in slab-geometry neutron transport. By drawing an analogy to Least-Squares Finite Elements (LSFE), the authors show that vanilla PINNs with ReLU activations fail to reproduce the correct diffusion limit as the mean free path becomes small, and they introduce a diffusive scaling (S = P + \tau(I - P)) to remedy this. Theoretical results, complemented by numerical experiments using PN angular discretization, demonstrate that the scaled PINN (and LSFE) solutions converge toward the diffusion solution across test cases, including interfaces and non-ReLU activations like tanh. The findings highlight the importance of asymptotic-preserving scalings for multiscale PDEs and suggest directions for formal convergence proofs and extensions to other activations and scalings.

Abstract

We investigate the numerical solution of multiscale transport equations using Physics Informed Neural Networks (PINNs) with ReLU activation functions. Therefore, we study the analogy between PINNs and Least-Squares Finite Elements (LSFE) which lies in the shared approach to reformulate the PDE solution as a minimization of a quadratic functional. We prove that in the diffusive regime, the correct limit is not reached, in agreement with known results for first-order LSFE. A diffusive scaling is introduced that can be applied to overcome this, again in full agreement with theoretical results for LSFE. We provide numerical results in the case of slab geometry that support our theoretical findings.

Paper Structure

This paper contains 13 sections, 2 theorems, 49 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Physics Informed Neural Networks
  3. Least-Squares Finite Elements
  4. PINNs in the asymptotic limit
  5. Characterization of the minimizer
  6. Scaling for the numerical solution
  7. Numerical Results
  8. Angular Discretization
  9. Implementation
  10. Asymptotic Test of the Diffusion Limit
  11. Diffusive Test with an Interface
  12. Diffusive Test with Hyperbolic Tangent Activation Functions
  13. Discussion

Key Result

Lemma 1

Let the Functional $F$ and the set of functions $V_h$ be as given in Eqs. (eq:min_F) and (eq:v_h). Suppose $\Psi^\varepsilon_{min}$ minimizes $F$ restricted to $V_h$. Suppose further that $\varepsilon\leq 1$ and that $\Psi^\varepsilon_{min}$ has an expansion with where $\eta_\nu$ and $\delta_\nu$ are independent of $\epsilon$. Then we have:

Figures (7)

  • Figure 1: In slab geometry, transport is projected onto the x-axis. The problem is symmetric in the y/z-plane.
  • Figure 2: Results for PINN with ReLU activation functions and first order LSFE in the asymptotic limit ($\varepsilon=10^{-4}$) compared to a MC reference solution. The setting is discussed in detail in Subsec. \ref{['subsec:diff_limit']}.
  • Figure 3: $\varepsilon=10^{-2}$
  • Figure 4: $\varepsilon=10^{-3}$
  • Figure 5: $\varepsilon=10^{-4}$
  • ...and 2 more figures

Theorems & Definitions (2)

  • Lemma 1
  • Theorem 1