Resolving Sharp Gradients of Unstable Singularities to Machine Precision via Neural Networks

TL;DR

This work tackles the daunting task of resolving highly unstable self-similar singularities in nonlinear PDEs by introducing a gradient-normalized PDE residual within a MSNN PINN framework. The method reweights residuals by local gradient magnitude to achieve uniform relative error, enabling convergence to machine precision for unstable CCF and IPM solutions and revealing a new 4th IPM unstable solution; it also extends to unstable NLS coherent structures, including GP vortices up to n=30 and excited states, where new empirical laws are discovered. The combination of gradient normalization, adaptive sampling, and multi-stage refinement produces robust, high-precision solutions that bridge numerical discovery and computer-assisted proofs, and suggests broad applicability to challenging PDE problems with sharp gradients. Overall, the approach provides a principled pathway to accurately identify self-similar scaling parameters and to uncover subtle local features essential for rigorous analysis of unstable nonlinear phenomena.

Abstract

Recent work introduced a robust computational framework combining embedded mathematical structures, advanced optimization, and neural network architecture, leading to the discovery of multiple unstable self-similar solutions for key fluid dynamics equations, including the Incompressible Porous Media (IPM) and 2D Boussinesq systems. While this framework confirmed the existence of these singularities, an accuracy level approaching double-float machine precision was only achieved for stable and 1st unstable solutions of the 1D Córdoba-Córdoba-Fontelos model. For highly unstable solutions characterized by extreme gradients, the accuracy remained insufficient for validation. The primary obstacle is the presence of sharp solution gradients. Those gradients tend to induce large, localized PDE residuals during training, which not only hinder convergence, but also obscure the subtle signals near the origin required to identify the correct self-similar scaling parameter lambda of the solutions. In this work, we introduce a gradient-normalized PDE residual re-weighting scheme to resolve the high-gradient challenge while amplifying the critical residual signals at the origin for lambda identification. Coupled with the multi-stage neural network architecture, the PDE residuals are reduced to the level of round-off error across a wide spectrum of unstable self-similar singularities previously discovered. Furthermore, our method enables the discovery of new highly unstable singularities, i.e. the 4th unstable solution for IPM equations and a novel family of highly unstable solitons for the Nonlinear Schrödinger equations. This results in achieving high-gradient solutions with high precision, providing an important ingredient for bridging the gap between numerical discovery and computer-assisted proofs for unstable phenomena in nonlinear PDEs.

Figures (12)

• Figure 1: High-gradient feature of unstable solutions (a) Comparison of stable and unstable solution for (a) CCF equation and (b) IPM equations. The highly unstable solution shows significant higher local gradient than that of the stable or the low-order unstable solutions.
• Figure 2: Challenges of 2nd unstable solutions to CCF (a) Comparison of the PDE residual for training the first and second unstable solutions for the CCF equation using the PINN framework with uniform weighting. (b & c) The PDE residual of 2nd unstable solution trained by (b) increase or (c) reduce the weights of collocation points around the high-gradient area of the solution ($y=4$). (d) The spatial profile of the solutions $\Omega$ and $U$ for the 2nd unstable solution with the inferred $\lambda_s = 0.47132422$ by a single-stage training using gradient-normalized residual. (e & f) The (e) absolute and (f) relative PDE residual for the CCF solution trained at the fixed $\lambda$ with $10^{-7}$ distance to the inferred smooth $\lambda_s$.
• Figure 3: CCF solutions with low $\lambda$. (a) Comparison of smooth and non-smooth CCF solutions for $\lambda < 0.61$. The magnitude and the sharpness of the high-gradient areas increase with the decrease of $\lambda$ (b). Evolution of the peak value of the CCF solution with respect to $\lambda$, illustrating the intensification of gradients as instability increases. The inset shows that PDE residual for the CCF solution at the fixed $\lambda = 0.455$, which has non-smooth signal at the origin, indicating that it remains a non-smooth solution.
• Figure 4: Challenge of 4th unstable solution to IPM. (a) The vorticity profile $\Omega$ for the 4th unstable solution to IPM with the inferred smooth $\lambda_s$. (b & c) The PDE residual from the PINN training using standard equation loss at two different $\lambda$ around the 4th unstable solution, where the spurious non-smooth signals at the origin are nearly identical. (d) The difference between the solutions $H^{(1)}$ and $H^{(2)}$, which are trained at the same fixed $\lambda = 0.1990$ via the same PINN framework using standard equation loss but different random initialization of the network weight. The solution discrepancy $O(10^{-4})$ is much larger than the PDE residual $O(10^{-6})$ (panel b) for each solution. (e) In comparison, the difference between the solution using gradient-normalized residual remains the same order of magnitude with the PDE residual $O(10^{-8})$ (panel h). (f) The relation of the maximum value $R_{max}$ of the relative PDE residual at the origin captured via PINN framework using standard equation loss (red) and gradient-normalized residual (blue) with the distance of $\lambda$ to the true smooth $\lambda_s$. (g) the absolute PDE residual obtained via PINN framework using gradient-normalized residual. (h & i) The relative PDE residual normalized by the solution magnitude at two different $\lambda$, which reveals the true non-smooth signal at the origin that are proportional to the distance from the true smooth $\lambda$ (panel e).
• Figure 5: Multistage training on 2nd unstable CCF solutions (a) PDE residuals after second-stage training using the gradient-normalized residual based on the solution magnitude, which provide spurious non-smooth signal that remains identical for different $\lambda$ (b) Spurious signals for a fixed $\lambda$ are inconsistent, flipping sign between experiments with different random initializations (c) Comparison between the profiles of $\Omega$ and its derivative $\partial_y\Omega$. The latter has much higher gradient. (d) The solution error $\Omega_{err} = \Omega_g - \Omega_0$ between the ground-truth solution and the first-stage results $\Omega_0$ trained at different fixed $\lambda$. $\epsilon_\lambda$ indicates the proximity of the fixed $\lambda$ to the ground-truth$\lambda_s$ for the 2nd CCF unstable solution. Here, the ground-truth solution $\Omega_g$ and lambda $\lambda_s$ refer to the best 2nd CCF unstable solution obtained that reaches machine precision as shown in e. (e) The normalized solution error by the error $\epsilon_\lambda$ of $\lambda$. (f) The absolute PDE residual for the 2nd CCF unstable solution using the new gradient-normalized residual based on the derivative of the solution $\partial_y\Omega_0$. The peak in the region of high solution gradient is restricted by the round-off error of double-float precision. (g) The relative PDE residual normalized by the solution magnitude.