Preserving Conservation Laws in the Time-Evolving Natural Gradient Method via Relaxation and Projection Techniques

Abstract

Neural networks have demonstrated significant potential in solving partial differential equations (PDEs). While global approaches such as Physics-Informed Neural Networks (PINNs) offer promising capabilities, they often lack inherent temporal causality, which can limit their accuracy and stability for time-dependent problems. In contrast, local training frameworks that progressively update network parameters over time are naturally suited for evolving PDEs. However, a critical challenge remains: many physical systems possess intrinsic invariants -- such as energy or mass -- that must be preserved to ensure physically meaningful solutions. This paper addresses this challenge by enhancing the Time-Evolving Natural Gradient (TENG) method, a recently proposed local training framework. We introduce two complementary techniques: (i) a relaxation algorithm that ensures the target solution $u_{\text{target}}$ preserves both quadratic and general nonlinear invariants of the original system, providing a structure-preserving learning target; and (ii) a projection technique that maps the updated network parameters $θ(t)$ back onto the invariant manifold, ensuring the final neural network solution strictly adheres to the conservation laws. Numerical experiments on the inviscid Burgers equation, Korteweg-de Vries equation, and acoustic wave equation demonstrate that our proposed approach significantly improves conservation properties while maintaining high accuracy.

Figures (8)

• Figure 1: Evolution of the mass conservation error over time. The projection method (blue solid line) maintains the error near machine precision ($\sim 10^{-14}$), significantly outperforming the original method (orange dashed line) which exhibits fluctuations between $10^{-9}$ and $10^{-5}$.
• Figure 2: Numerical performance of the TENG framework on the inviscid Burgers' equation. (a) Mass conservation error relative to different time step sizes and the projection method. (b) Temporal evolution of the relative $L^2$ error, highlighting the accuracy gains from the projection-based physical constraint.
• Figure 3: Numerical performance of the TENG framework on the KdV equation. (a) Necessity of the relaxation method: The RP-TENG framework suppresses the quadratic invariant error $|\mathcal{H}(u_{\mathrm{target}}) - \mathcal{H}(\hat{u}_{\theta_{n}})|$ to machine epsilon ($\sim 10^{-15}$), effectively eliminating the numerical drift of the baseline. (b) Influence on dynamical trajectory: The stability of $\gamma \approx 1$ confirms that the energy constraint is enforced with minimal intrusiveness to the neural dynamical trajectory.
• Figure 4: Numerical performance of RP-TENG on the KdV equation. (a) Energy conservation: RP-TENG maintains the Hamiltonian error $|\mathcal{H}(u_0) - \mathcal{H}(\hat{u}_{\theta_{n}})|$ near machine epsilon, eliminating the drift of vanilla TENG. (b) Parameter stability: The temporal consistency of $\gamma \approx 1$ confirms the non-intrusiveness of the projection step within the dual-constraint framework.
• Figure 5: Comprehensive performance on the KdV equation soliton simulation. (a) Temporal evolution of the relative $L^2$ error against the analytical solution.(b) Long-term energy-preserving capability of the RP-TENG framework for $t \in [0, 10]$

Theorems & Definitions (1)

• Remark 2.1