Symplectic Structure-Aware Hamiltonian (Graph) Embeddings

TL;DR

This work tackles the rigidity of fixed embedding manifolds in Graph Neural Networks by introducing SAH-GNN, which generalizes Hamiltonian dynamics to graph embeddings with an adaptively learned symplectic structure. The core idea is to parameterize the symplectic transformation with matrices $M^{(l)}$ drawn from the symplectic group $\,\mathrm{Sp}(2k^{(l+1)},2k^{(l)})$ and optimize them via hard, Riemannian gradient descent on the symplectic Stiefel manifold, ensuring $M^\top J M = J$ and energy conservation throughout training. The paper provides random initialization for symplectic matrices, a projection-based tangent-space update, and a Cayley retraction to maintain symplecticity, enabling SAH-GNN to adapt to diverse graph geometries across datasets. Empirical results on node classification show SAH-GNN consistently outperforms or matches state-of-the-art methods while maintaining energy stability, indicating the practical value of learning a geometry-aware, physically meaningful embedding space. This approach opens avenues for broader applications of Hamiltonian NN frameworks where adaptive geometry and energy conservation are beneficial.

Abstract

In traditional Graph Neural Networks (GNNs), the assumption of a fixed embedding manifold often limits their adaptability to diverse graph geometries. Recently, Hamiltonian system-inspired GNNs have been proposed to address the dynamic nature of such embeddings by incorporating physical laws into node feature updates. We present Symplectic Structure-Aware Hamiltonian GNN (SAH-GNN), a novel approach that generalizes Hamiltonian dynamics for more flexible node feature updates. Unlike existing Hamiltonian approaches, SAH-GNN employs Riemannian optimization on the symplectic Stiefel manifold to adaptively learn the underlying symplectic structure, circumventing the limitations of existing Hamiltonian GNNs that rely on a pre-defined form of standard symplectic structure. This innovation allows SAH-GNN to automatically adapt to various graph datasets without extensive hyperparameter tuning. Moreover, it conserves energy during training meaning the implicit Hamiltonian system is physically meaningful. Finally, we empirically validate SAH-GNN's superiority and adaptability in node classification tasks across multiple types of graph datasets.

Figures (3)

• Figure 1: (a) Schematic of SAH-GNN. Essentially, it follows the process Eq. (\ref{['eq:hnn-integration']}-\ref{['hamgnn-agg']}), we do not change the MLP nature of $\mathcal{Q}^{(l)}_\phi$ and $\mathcal{H}^{(l)}_\theta$. (b) Different optimization method upon symplectic structural matrix $M$, where Riemannian gradient descent is leveraged by SAH-GNN during back-propagation.
• Figure 2: Variation of energy $\mathcal{H}_\theta$ through training via different optimization method of symplectic transformation $M$. A more stable curve is demanded as better conserved energy.
• Figure 3: Training losses on various datasets with 10 runs, we visualize the averaged curves with standard deviation.

Theorems & Definitions (14)

• Theorem 2.1: Conservation law da2001lectures
• proof
• Proposition 3.2: Preservation of Volume and Orientation
• proof
• Proposition 3.3: Conservation of Symplectic Form
• proof
• Proposition 3.4
• proof
• Definition 3.5: Symplectic projection operator
• Proposition 3.6