Learning Relativistic Geodesics and Chaotic Dynamics via Stabilized Lagrangian Neural Networks

TL;DR

This work addresses the instability of training Lagrangian Neural Networks (LNNs) by introducing Hessian-based regularization, gradient clipping, physics-aware scaling, and careful activation/initialization choices to enable learning of complex dynamics. The authors develop two regularization strategies—an eigenvalue penalty and a scale-invariant Sylvester-criterion approach—to enforce physically meaningful mass-matrix signatures for both classical and relativistic systems, including AdS$_4$ geodesics. They demonstrate substantial gains in stability and accuracy, achieving, for example, ~96% reductions in validation loss for double pendulum and enabling the learning of geodesic Lagrangians in non-relativistic and relativistic spacetimes, with potential to extract metric components from learned mass matrices. While limitations remain (notably the need for invertible Hessians and handling of coordinate singularities), the approach broadens LNN applicability to automated discovery of geometric structures and spacetime metrics from trajectory data, with broad implications for physics-informed machine learning and scientific discovery.

Abstract

Lagrangian Neural Networks (LNNs) can learn arbitrary Lagrangians from trajectory data, but their unusual optimization objective leads to significant training instabilities that limit their application to complex systems. We propose several improvements that address these fundamental challenges, namely, a Hessian regularization scheme that penalizes unphysical signatures in the Lagrangian's second derivatives with respect to velocities, preventing the network from learning unstable dynamics, activation functions that are better suited to the problem of learning Lagrangians, and a physics-aware coordinate scaling that improves stability. We systematically evaluate these techniques alongside previously proposed methods for improving stability. Our improved architecture successfully trains on systems of unprecedented complexity, including triple pendulums, and achieved 96.6\% lower validation loss value and 90.68\% better stability than baseline LNNs in double pendulum systems. With the improved framework, we show that our LNNs can learn Lagrangians representing geodesic motion in both non-relativistic and general relativistic settings. To deal with the relativistic setting, we extended our regularization to penalize violations of Lorentzian signatures, which allowed us to predict a geodesic Lagrangian under AdS\textsubscript{4} spacetime metric directly from trajectory data, which to our knowledge has not been done in the literature before. This opens new possibilities for automated discovery of geometric structures in physics, including extraction of spacetime metric tensor components from geodesic trajectories. While our approach inherits some limitations of the original LNN framework, particularly the requirement for invertible Hessians, it significantly expands the practical applicability of LNNs for scientific discovery tasks.

Figures (26)

• Figure 1: Comparison of training loss with and without regularization. Both eigenvalue and Sylvester's condition regularization methods enable successful training where unregularized LNN are unstable and fail more often.
• Figure 2: Performance comparison of physical scaling vs. no scaling vs. DyT normalization.
• Figure 3: Trade-off between validation loss and training instability, with and without an initial velocity term.
• Figure 4: Comparison of learned mass matrix behavior of a double pendulum across different training ranges. The $[0, 2\pi]$ range exhibits problematic determinant signs near boundaries (left), while the extended $[-\pi/2, 2\pi + \pi/2]$ range exhibits similar shape to the analytical Hessian and produces correct behavior throughout the domain (right). An analytical determinant of the Hessian matrix is provided at the center.
• Figure 5: Performance vs Training Instability Trade-off between two different proposed models in the literature.