Learning solutions to some toy constrained optimization problems in infinite dimensional Hilbert spaces

TL;DR

The paper addresses the challenge of solving constrained optimization problems in infinite-dimensional Hilbert spaces by implementing deep-learning variants of the penalty and augmented Lagrangian methods. It frames problems as minimizing $f(u)$ subject to $g(u)=0$ and tests two neural-network-parameterized approaches to solve the resulting subproblems, with separate handling for finite and infinite-dimensional constraint outputs. Across four toy problems from calculus of variations and physics—minimal surfaces, geodesics, Grad-Shafranov, and Beltrami fields—it reports that both methods yield decently accurate solutions and comparable error profiles, while the augmented Lagrangian approach provides notable speedups when the constraint output space $W$ is infinite-dimensional, due to cheaper multiplier updates. The work demonstrates a practical pathway to couple classical infinite-dimensional optimization with neural approximations, suggesting further exploration of multiplier-update variants and loss-landscape characteristics to enhance efficiency and stability in more complex variational problems.

Abstract

In this work we present deep learning implementations of two popular theoretical constrained optimization algorithms in infinite dimensional Hilbert spaces, namely, the penalty and the augmented Lagrangian methods. We test these algorithms on some toy problems originating in either calculus of variations or physics. We demonstrate that both methods are able to produce decent approximations for the test problems and are comparable in terms of different errors produced. Leveraging the common occurrence of the Lagrange multiplier update rule being computationally less expensive than solving subproblems in the penalty method, we achieve significant speedups in cases when the output of the constraint function is itself a function.

Figures (9)

• Figure 2: Solutions to the minimal surface problem. Darker color implies higher $u$ value.
• Figure 3: Errors and run times for the minimal surface problem as functions of gradient descent steps. The errors have been plotted in a semilog fashion. All quantities have been plotted every 100 steps.
• Figure 4: Solutions to the geodesic problem. The distance between the black dots is being minimized.
• Figure 5: Errors and run times for the geodesic problem as functions of gradient descent steps. The errors have been plotted in a semilog fashion. All quantities have been plotted every 100 steps.
• Figure 6: Solutions to the geodesic problem when the points (black dots) are antipodal