Gauss-Newton Natural Gradient Descent for Shape Learning

TL;DR

This work tackles ill-conditioning in geometry-constrained shape learning by transferring optimization to a function-space perspective and applying Gauss-Newton natural gradient descent (GN-NGD). It reformulates geometric losses as $L^2$ residuals, derives a Gauss-Newton Gramian, and develops scalable, matrix-free and Woodbury-based updates suitable for moving-surface losses. Across Plateau's problem, developable surfaces, implicit neural shapes, and a jet-engine-bracket design task, GN-NGD consistently achieves faster convergence and lower errors than Adam or LBFGS, including in data-sparse scenarios. The results indicate that incorporating second-order, curvature-aware updates in function space can substantially improve the efficiency and accuracy of shape learning, with potential impact on physics-informed modeling and computational design.

Abstract

We explore the use of the Gauss-Newton method for optimization in shape learning, including implicit neural surfaces and geometry-informed neural networks. The method addresses key challenges in shape learning, such as the ill-conditioning of the underlying differential constraints and the mismatch between the optimization problem in parameter space and the function space where the problem is naturally posed. This leads to significantly faster and more stable convergence than standard first-order methods, while also requiring far fewer iterations. Experiments across benchmark shape optimization tasks demonstrate that the Gauss-Newton method consistently improves both training speed and final solution accuracy.

Figures (9)

• Figure 1: The absolute mean-curvature $\left\lvert\kappa_H(\mathbf{x})\right\rvert$ plotted on the best model surface trained with Adam (left), LBFGS (middle), Gauss-Newton (right) for 20 minutes.
• Figure 2: Comparison of Loss (interface + mean curvature) and Chamfer Discrepancy Over Time for the Catenoid.
• Figure 3: $\left\lvert\kappa_H(\mathbf{x})\right\rvert$ plotted on the best model surface trained with Adam (left) and Gauss-Newton (right) for 20 minutes. LBFGS becomes unstable.
• Figure 4: Comparison of Loss (interface + mean curvature) and Chamfer Discrepancy Over Time for the Enneper Surface.
• Figure 5: Comparison of Adam and Gauss-Newton optimization on the truncated cone problem. (a) The absolute Gauss curvature $\left\lvert\kappa_G(\mathbf{x})\right\rvert$ is plotted for the best model surface trained with Adam (left) and Gauss-Newton (right) for 20 minutes. LBFGS becomes unstable and is omitted. (b) Total loss (interface + Gauss-curvature) over time for the truncated cone.