Improved Initialization for Port-Hamiltonian Neural Network Models

TL;DR

An improved initialization for port-Hamiltonian neural networks is proposed, which first estimates a linear port-Hamiltonian system to be used as an initialization for the network, after which the neural network adapts to the system nonlinearities, reducing the training times and improving convergence.

Abstract

Port-Hamiltonian neural networks have shown promising results in the identification of nonlinear dynamics of complex systems, as their combination of physical principles with data-driven learning allows for accurate modelling. However, due to the non-convex optimization problem inherent in learning the correct network parameters, the training procedure is prone to converging to local minima, potentially leading to poor performance. In order to avoid this issue, this paper proposes an improved initialization for port-Hamiltonian neural networks. The core idea is to first estimate a linear port-Hamiltonian system to be used as an initialization for the network, after which the neural network adapts to the system nonlinearities, reducing the training times and improving convergence. The effectiveness of this method is tested on a chained mass-spring-damper setup for varying noise levels and compared to the original approach.

Figures (5)

• Figure 1: Schematic overview of the model structure including the linear PH estimate.
• Figure 2: Schematic view of the three coupled mass-spring-dampers. For this system, the input is given as a force $F_{1}$, the states are the displacements, ${q}(t)$, and momenta, ${p}(t)$, of the masses, while the outputs are the velocities of the masses $\dot{{q}}(t)$. Note that the dampers have both a linear and nonlinear component.
• Figure 3: Model convergence during the training process. Yellow shows the linear model, red the nonlinear model and blue the nonlinear model with the linear initialization. The mean for each model class is shown in bold, with individual models opaque.
• Figure 4: Qualitative simulation of the trained models on a segment of the test dataset. Model class averages are given by the dashed lines, while the shaded areas represent the standard deviations.
• Figure 5: Boxplot showing the spread in model NRMSE's across the considered model classes. The dashed line indicates the noise floor, which is at a SNR of 30dB in this case.