The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems

TL;DR

The paper introduces the Deep Ritz Method, a neural network-based approach to variational formulations of PDEs, representing trial functions with deep networks and optimizing the Ritz-type functional via stochastic gradient descent with domain sampling. By embedding residual blocks and a tailored activation, it achieves adaptive, scalable solutions across low and high dimensions, demonstrated on Poisson problems, Neumann cases, transfer learning scenarios, and eigenvalue problems. The results show promising accuracy with relatively small networks and highlight the method's potential for high-dimensional PDEs, while noting challenges in nonconvex optimization and boundary condition handling. Overall, it offers a new variationally grounded, machine-learning-compatible framework for PDE numerics with notable adaptability and room for methodological refinements.

Abstract

We propose a deep learning based method, the Deep Ritz Method, for numerically solving variational problems, particularly the ones that arise from partial differential equations. The Deep Ritz method is naturally nonlinear, naturally adaptive and has the potential to work in rather high dimensions. The framework is quite simple and fits well with the stochastic gradient descent method used in deep learning. We illustrate the method on several problems including some eigenvalue problems.

Figures (6)

• Figure 1: The figure shows a network with two blocks and an output linear layer. Each block consists of two fully-connected layers and a skip connection.
• Figure 2: Solutions computed by two different methods.
• Figure 3: The total error and error at the boundary during the training process. The x-axis represents the iteration steps. The blue curves show the relative error of $u$. The red curves show the relative error on the boundary.
• Figure 4: The error during the training process ($d=5$ and $d=10$).
• Figure 5: The red curves show the results of the training process with weight transfer. The blue curves show the results of the training process with random initialization. The left figure shows how the natural logarithm of the error changes during training. The right figure shows how the natural logarithm of $||\Delta W||^2_2$ changes during training, where $\Delta W$ is the change in $W$ after 100 training steps, $W$ is the weight matrix.