Frequency-Separable Hamiltonian Neural Network for Multi-Timescale Dynamics

TL;DR

The Frequency-Separable Hamiltonian Neural Network is introduced, which parameterizes the system Hamiltonian using multiple networks, each governed by Hamiltonian dynamics and trained on data sampled at distinct timescales, and extends this framework to partial differential equations by learning a state- and boundary-conditioned symplectic operators.

Abstract

While Hamiltonian mechanics provides a powerful inductive bias for neural networks modeling dynamical systems, Hamiltonian Neural Networks and their variants often fail to capture complex temporal dynamics spanning multiple timescales. This limitation is commonly linked to the spectral bias of deep neural networks, which favors learning low-frequency, slow-varying dynamics. Prior approaches have sought to address this issue through symplectic integration schemes that enforce energy conservation or by incorporating geometric constraints to impose structure on the configuration-space. However, such methods either remain limited in their ability to fully capture multiscale dynamics or require substantial domain specific assumptions. In this work, we exploit the observation that Hamiltonian functions admit decompositions into explicit fast and slow modes and can be reconstructed from these components. We introduce the Frequency-Separable Hamiltonian Neural Network (FS-HNN), which parameterizes the system Hamiltonian using multiple networks, each governed by Hamiltonian dynamics and trained on data sampled at distinct timescales. We further extend this framework to partial differential equations by learning a state- and boundary-conditioned symplectic operators. Empirically, we show that FS-HNN improves long-horizon extrapolation performance on challenging dynamical systems and generalizes across a broad range of ODE and PDE problems.

Figures (7)

• Figure 1: Frequency-Separable Hamiltonian Neural Network. Hamiltonian systems that exhibit dynamics with multiple timescales and be decoupled into subsystems each representing one timescale. An observation $u_{\mathrm{train}}$ is subsampled using interval $I_{1}$, $I_{2}$, and $I_{3}$ to form subsampled dataset $u^{(k)}$. The single scale HNN $\mathcal{M}_{k}$ is trained on subsampled data individually. The multiscale model $\mathcal{M}_{\mathrm{multi}}$ combines pretrained single scale HNN trajectories to predict rollout on the original resolution.
• Figure 2: Phase space comparison for ODE systems. We show the MSE error and phase space portraits of the predicted rollout of FS-HNN, MLP, HNN greydanus2019hamiltonian, and SympNet jin2020sympnets for the (Top) ideal pendulum, (Middle) double pendulum, and (Bottom) Fermi-Pasta-Uam-Tsingou systems. FS-HNN achieves significantly lower prediction error compared to baseline networks. Some baselines are omitted from the MSE plot due to excessively large predictive error. Exact trajetocy prediction results can be checked in Figure \ref{['fig:odecomparison']} in Appendix \ref{['oderesults']}.
• Figure 3: Flow field comparison for PDE systems. We report the rollout MSE error of FS-HNN, PHNN eidnes2024pseudo, and FNO li2020fourier on the (Top) SWE with Gaussian pulse initialization, (Middle) Taylor-Green vortex systems, and (Bottom) SWE with random initialization. FS-HNN learns qualitatively similar flow fields and achieves significantly lower prediction error compared to baseline networks. Results are plotted for the potential height(SWE)/pressure distribution(Taylor-Green) channel, results for the remaining states are given in Figure \ref{['fig:pdesup']} in Appendix \ref{['pderesults']}.
• Figure 4: Relative energy deviation in ODE settings. The energy is not strictly conserved; instead, it shows a small periodic perturbation. Even so, FS-HNN matches the ground truth most closely.
• Figure 5: Relative energy deviation in PDE settings. Due to the dataset setup, energy is not strictly conserved; however, the variation remains small. We therefore report energy changes in relative terms. Overall, FS-HNN matches the ground truth most closely