A practical PINN framework for multi-scale problems with multi-magnitude loss terms

TL;DR

This work tackles the difficulty of solving multi-scale PDEs with PINNs by addressing loss-term magnitude disparities and high-frequency content. It introduces the MMPINN framework, which regularizes loss terms via $ ilde{\mathcal{L}}=w_s\mathcal{L}_s^{1/m}+w_r\mathcal{L}_r^{1/n}$, employs multi-level training, and uses grouping regularization and integrated neural network architectures (INN and Fourier features) to handle multi-frequency behavior. Across heat, Poisson, Helmholtz, Klein–Gordon, and multi-frequency problems, MMPINN variants (DNN, MFF, INN) achieve substantially improved accuracy and convergence compared to conventional PINNs and recent baselines, often by orders of magnitude in $L_2$ relative error. The framework offers a practical, unified pathway to apply PINNs to challenging multi-scale problems with high-frequency content, with clear avenues for adaptive parameter selection and architecture optimization in future work.

Abstract

For multi-scale problems, the conventional physics-informed neural networks (PINNs) face some challenges in obtaining available predictions. In this paper, based on PINNs, we propose a practical deep learning framework for multi-scale problems by reconstructing the loss function and associating it with special neural network architectures. New PINN methods derived from the improved PINN framework differ from the conventional PINN method mainly in two aspects. First, the new methods use a novel loss function by modifying the standard loss function through a (grouping) regularization strategy. The regularization strategy implements a different power operation on each loss term so that all loss terms composing the loss function are of approximately the same order of magnitude, which makes all loss terms be optimized synchronously during the optimization process. Second, for the multi-frequency or high-frequency problems, in addition to using the modified loss function, new methods upgrade the neural network architecture from the common fully-connected neural network to special network architectures such as the Fourier feature architecture, and the integrated architecture developed by us. The combination of the above two techniques leads to a significant improvement in the computational accuracy of multi-scale problems. Several challenging numerical examples demonstrate the effectiveness of the proposed methods. The proposed methods not only significantly outperform the conventional PINN method in terms of computational efficiency and computational accuracy, but also compare favorably with the state-of-the-art methods in the recent literature. The improved PINN framework facilitates better application of PINNs to multi-scale problems.

Figures (17)

• Figure 3.1: Grouping along the time direction.
• Figure 3.2: INN network architecture.
• Figure 3.3: Diagram of the MMPINN framework.
• Figure 4.1: The exact solution of Eq. \ref{['hcp1']} with $\varepsilon=0.15$ and the point-wise absolute errors of the SA-PINN Self-adaptivepinnjcp and MMPINN-DNN methods.
• Figure 4.2: Loss curves of the conventional PINN method and the MMPINN-DNN method for solving Eq. \ref{['hcp1']} with $\varepsilon=0.15$.

Theorems & Definitions (1)

• Remark 1