Symplectic Methods in Deep Learning
Sofya Maslovskaya, Sina Ober-Blöbaum
TL;DR
The paper addresses safe, theory‑grounded deep learning by leveraging symplectic structure to preserve Hamiltonian flows and stabilize gradient propagation. It introduces SPRK Net, an explicit higher‑order neural architecture built from symplectic partitioned Runge‑Kutta methods that guarantees symplecticity, a non‑vanishing gradient, and universal approximation. Empirical results on image classification and autonomous Hamiltonian system learning show that higher‑order SPRK nets outperform previous Hamiltonian networks under similar parameter budgets. The work bridges discrete network design with continuous‑time optimal control and opens avenues for reversible, structure‑preserving learning architectures.
Abstract
Deep learning is widely used in tasks including image recognition and generation, in learning dynamical systems from data and many more. It is important to construct learning architectures with theoretical guarantees to permit safety in the applications. There has been considerable progress in this direction lately. In particular, symplectic networks were shown to have the non vanishing gradient property, essential for numerical stability. On the other hand, architectures based on higher order numerical methods were shown to be efficient in many tasks where the learned function has an underlying dynamical structure. In this work we construct symplectic networks based on higher order explicit methods with non vanishing gradient property and test their efficiency on various examples.