Hopfield neural networks as port-Hamiltonian and gradient systems
Arjan van der Schaft
TL;DR
This paper reinterprets continuous Hopfield networks as nonlinear capacitors in a generalized electrical network, showing that they admit a port-Hamiltonian formulation under a passivity condition and can be represented as gradient systems with a Hessian-based Riemannian metric. It clarifies that the traditional Hopfield energy plays the role of a dissipation potential, not a total energy, and derives a corresponding dissipation inequality to support analysis and interconnection. For constant inputs, the gradient formulation implies convergence to local minima of the dissipation potential, with sigmoid activations bounding the state space and yielding associative-memory behavior. Overall, the work links port-Hamiltonian and gradient frameworks for Hopfield networks, providing a rigorous foundation for interconnection and control of such networks.
Abstract
The structure of continuous Hopfield networks is revisited from a system-theoretic point of view. After adopting a novel electrical network interpretation involving nonlinear capacitors, it is shown that Hopfield networks admit a port-Hamiltonian formulation provided an extra passivity condition is satisfied. Subsequently it is shown that any Hopfield network can be represented as a gradient system, with Riemannian metric given by the inverse of the Hessian matrix of the total energy stored in the nonlinear capacitors. On the other hand, the well-known 'energy' function employed by Hopfield turns out to be the dissipation potential of the gradient system, and this potential is shown to satisfy a dissipation inequality that can be used for analysis and interconnection.