Regularity-Conforming Neural Networks (ReCoNNs) for solving Partial Differential Equations

TL;DR

This work tackles solving PDEs with low-regularity solutions where standard neural networks struggle to capture gradient discontinuities and singularities. It introduces Regularity-Conforming Neural Networks (ReCoNNs), which encode a priori solution regularity—via interface terms and singular units—into the architecture, enabling strong-form losses that incorporate explicit interface and flux conditions. The authors design architectures for both gradient jumps without point singularities and for point singularities at re-entrant corners and material-vertex intersections, demonstrating superior accuracy and training stability compared to classical networks on 1D and 2D transmission problems (including L-shaped domains). The results show that ReCoNNs reduce Gibbs phenomena and instability near singularities, provide partial explainability of the learned singular structures, and hold promise for extension to higher dimensions and inverse problems in engineering contexts.

Abstract

Whilst the Universal Approximation Theorem guarantees the existence of approximations to Sobolev functions -- the natural function spaces for PDEs -- by Neural Networks (NNs) of sufficient size, low-regularity solutions may lead to poor approximations in practice. For example, classical fully-connected feed-forward NNs fail to approximate continuous functions whose gradient is discontinuous when employing strong formulations like in Physics Informed Neural Networks (PINNs). In this article, we propose the use of regularity-conforming neural networks, where a priori information on the regularity of solutions to PDEs can be employed to construct proper architectures. We illustrate the potential of such architectures via a two-dimensional (2D) transmission problem, where the solution may admit discontinuities in the gradient across interfaces, as well as power-like singularities at certain points. In particular, we formulate the weak transmission problem in a PINNs-like strong formulation with interface and continuity conditions. Such architectures are partially explainable; discontinuities are explicitly described, allowing the introduction of novel terms into the loss function. We demonstrate via several model problems in one and two dimensions the advantages of using regularity-conforming architectures in contrast to classical architectures. The ideas presented in this article easily extend to problems in higher dimensions.

Figures (25)

• Figure 1: The architecture of a fully-connected feed-forward neural network. The trainable parameters are contained within the grey box.
• Figure 2: Classical architecture with Monte Carlo integration and tanh activation function.
• Figure 3: Classical architecture with Monte Carlo integration and ReLu activation function.
• Figure 4: Regularity-conforming architecture with Monte Carlo integration and $tanh$ activation function.
• Figure 5: Examples of geometries that permit point singularities.