TENG++: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets under General Boundary Conditions
Xinjie He, Chenggong Zhang
TL;DR
This work addresses solving PDEs with non-periodic boundary conditions using neural nets. It extends the Time-Evolving Natural Gradient (TENG) framework to Dirichlet boundaries by adding boundary penalties and integrating Euler/Heun time-stepping. Experiments on the heat equation show Heun's higher accuracy and reveal initialization sensitivity as a key factor for performance. The approach lays groundwork for extending to Neumann and mixed boundary conditions and broader PDE classes, expanding the applicability of neural PDE solvers to real-world problems.
Abstract
Partial Differential Equations (PDEs) are central to modeling complex systems across physical, biological, and engineering domains, yet traditional numerical methods often struggle with high-dimensional or complex problems. Physics-Informed Neural Networks (PINNs) have emerged as an efficient alternative by embedding physics-based constraints into deep learning frameworks, but they face challenges in achieving high accuracy and handling complex boundary conditions. In this work, we extend the Time-Evolving Natural Gradient (TENG) framework to address Dirichlet boundary conditions, integrating natural gradient optimization with numerical time-stepping schemes, including Euler and Heun methods, to ensure both stability and accuracy. By incorporating boundary condition penalty terms into the loss function, the proposed approach enables precise enforcement of Dirichlet constraints. Experiments on the heat equation demonstrate the superior accuracy of the Heun method due to its second-order corrections and the computational efficiency of the Euler method for simpler scenarios. This work establishes a foundation for extending the framework to Neumann and mixed boundary conditions, as well as broader classes of PDEs, advancing the applicability of neural network-based solvers for real-world problems.