Auto-Adaptive PINNs with Applications to Phase Transitions

TL;DR

This work tackles training PINNs for time-dependent PDEs by introducing auto-adaptive sampling that can depend on the network and its derivatives. Focusing on the Allen-Cahn equation, the authors develop an energy-adaptive sampling strategy by coupling Metropolis-Hastings sampling with a heuristic based on the pointwise energy density, enabling targeted refinement near evolving interfaces. They integrate this with time-slicing, learning-rate schedules, and standard PINN techniques, and show that energy-adaptive PINNs outperform residual-adaptive baselines in capturing sharp interfaces and slow-evolving regions, while addressing issues like catastrophic unlearning seen in some time-sliced approaches. The method promises broader applicability to gradient-flow problems and offers a flexible framework for problem-specific sampling without manual post-hoc intervention, potentially advancing the practical deployment of PINNs in complex, multi-scale PDEs.

Abstract

We propose an adaptive sampling method for the training of Physics Informed Neural Networks (PINNs) which allows for sampling based on an arbitrary problem-specific heuristic which may depend on the network and its gradients. In particular we focus our analysis on the Allen-Cahn equations, attempting to accurately resolve the characteristic interfacial regions using a PINN without any post-hoc resampling. In experiments, we show the effectiveness of these methods over residual-adaptive frameworks.

Figures (8)

• Figure 1: Plots of the $L^2$ (left) and $L^\infty$ (right) error at the final simulation time as a function of $\lambda$, the proportion of adaptively sampled points. Each plot depicts the error of the residual adaptive in blue and the energy adaptive in orange.
• Figure 2: Plot of the loss against the number of ADAM iterations performed. The vertical axis represents the base $10$ log of the loss value, while the horizontal axis represents the number of epochs trained. The vertical red lines represent the increasing of the trained time domain, which is labeled at the top of the graph. Note every other red line also corresponds with a reduction in learning rate.
• Figure 3: Time slices from networks which have completed the described training. We present time slices from the the residual adaptive method and the energy adaptive method at times $0$, $.25$, $.5$, and $1$.
• Figure 4: Scatterplots containing the adaptively sampled points for the residual adaptive method (left) and the energy adaptive method (right). The horizontal axis represents the spatial domain and the vertical axis represents the temporal domain. These points are taken from near the end of network training and are thus representative of the entire spatiotemporal domain.
• Figure 5: We see the loss decreasing for each of the tested methods on the second example. The vertical axis represents the base $10$ log of the loss value, while the horizontal axis represents the number of epochs trained. The vertical red lines represent the increasing of the trained time domain, which is labeled at the top of the graph.