Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning

TL;DR

The paper tackles the training bottlenecks of physics-informed neural networks (PINNs) for high-frequency and multi-scale PDEs by introducing transfer learning from low-frequency baselines. It formalizes the PINN loss $L_{ ext{PINN}}( heta) = W_F L_F + W_I L_I + W_B L_B$ and compares optimizers like Adam and LBFGS. Through two case studies—the damped harmonic oscillator and the 1D wave equation—it demonstrates that transfer learning substantially accelerates convergence and reduces data needs without adding network capacity, with optimizer choice influencing transfer effectiveness. These findings offer practical guidelines for base-model selection and optimization, enabling PINNs to tackle more complex, multi-scale physical problems.

Abstract

Physics-informed neural network (PINN) is a data-driven solver for partial and ordinary differential equations(ODEs/PDEs). It provides a unified framework to address both forward and inverse problems. However, the complexity of the objective function often leads to training failures. This issue is particularly prominent when solving high-frequency and multi-scale problems. We proposed using transfer learning to boost the robustness and convergence of training PINN, starting training from low-frequency problems and gradually approaching high-frequency problems. Through two case studies, we discovered that transfer learning can effectively train PINN to approximate solutions from low-frequency problems to high-frequency problems without increasing network parameters. Furthermore, it requires fewer data points and less training time. We elaborately described our training strategy, including optimizer selection, and suggested guidelines for using transfer learning to train neural networks for solving more complex problems.

Figures (9)

• Figure 1: PINN Model
• Figure 2: Exact solution for different values of $\omega$
• Figure 3: Comparisons of Adam optimizer vs. L-BFGS optimizer at 20Hz
• Figure 4: Comparison of the Adam optimizer with the L-BFGS optimizer at 30Hz: (a) comparing the PINN solution and the exact solution using L-BFGS, (b) visualization of loss curve with L-BFGS Optimizer across epochs, (c) analyzing the PINN solution and exact solution using Adam, (d) Loss using Adam Optimizer.
• Figure 5: Comparisons of Adam optimizer vs. L-BFGS optimizer at 40Hz