Convergence of the generalization error for deep gradient flow methods for PDEs
Chenguang Liu, Antonis Papapantoleon, Jasper Rou
TL;DR
This work provides a rigorous convergence theory for deep gradient flow methods (DGFMs) in solving high‑dimensional PDEs by decoupling the generalization error into approximation, quadrature, and training components. It formulates the PDE as a variational energy minimization, proves well‑posedness and stable time discretization, and establishes a tailored universal approximation theorem to show neural networks can approximate PDE solutions in $\mathcal{H}_0^1(\mathbb{R}^d)$. The training analysis leverages a wide‑network (large width) gradient flow, deriving a kernel‑based limit and proving convergence of the trained network to the PDE solution, and subsequently shows long‑time convergence to the global minimizer of the loss. Collectively, these results imply that the generalization error tends to zero as the network width and training time grow, under verifiable assumptions on the PDE and network. The framework further clarifies the role of discretization and provides a pathway to rigorous guarantees for DGFMs in high‑dimensional settings.
Abstract
The aim of this article is to provide a firm mathematical foundation for the application of deep gradient flow methods (DGFMs) for the solution of (high-dimensional) partial differential equations (PDEs). We decompose the generalization error of DGFMs into an approximation and a training error. We first show that the solution of PDEs that satisfy reasonable and verifiable assumptions can be approximated by neural networks, thus the approximation error tends to zero as the number of neurons tends to infinity. Then, we derive the gradient flow that the training process follows in the ``wide network limit'' and analyze the limit of this flow as the training time tends to infinity. These results combined show that the generalization error of DGFMs tends to zero as the number of neurons and the training time tend to infinity.