A comparison of Single- and Double-generator formalisms for Thermodynamics-Informed Neural Networks

TL;DR

The paper investigates two physics-informed neural network formalisms that enforce thermodynamic consistency: a single‑generator bracket with a single energy potential $\mathcal{F}$, and GENERIC with two generators $(\mathcal{H},\mathcal{S})$ and degeneracy constraints ensuring $\frac{d\mathcal{H}}{dt}=0$ and $\frac{d\mathcal{S}}{dt}\ge 0$. Using Structure Preserving Neural Networks (SPNNs), it learns the appropriate operators $\boldsymbol{L},\boldsymbol{M}$ (via $\boldsymbol{l},\boldsymbol{m}$) and the energy generators, training with a loss that combines data fidelity, degeneracy constraints, and regularization. Two benchmark problems are studied: a double thermoelastic pendulum with Hamiltonian and dissipative couplings, and Couette flow of an Oldroyd‑B fluid with micro‑macro multiscale dynamics. The results show GENERIC improves accuracy by about an order of magnitude in some cases, but the single generator often achieves lower data and energy errors and can be more efficient, while degeneracy conditions in GENERIC offer robustness and better generalization under data scarcity or chaotic dynamics. Overall, there is no universally superior approach; the choice depends on the balance between dissipative versus conservative dynamics and on available data and computational resources.

Abstract

The development of inductive biases has been shown to be a very effective way to increase the accuracy and robustness of neural networks, particularly when they are used to predict physical phenomena. These biases significantly increase the certainty of predictions, decrease the error made and allow considerably smaller datasets to be used. There are a multitude of methods in the literature to develop these biases. One of the most effective ways, when dealing with physical phenomena, is to introduce physical principles of recognised validity into the network architecture. The problem becomes more complex without knowledge of the physical principles governing the phenomena under study. A very interesting possibility then is to turn to the principles of thermodynamics, which are universally valid, regardless of the level of abstraction of the description sought for the phenomenon under study. To ensure compliance with the principles of thermodynamics, there are formulations that have a long tradition in many branches of science. In the field of rheology, for example, two main types of formalisms are used to ensure compliance with these principles: one-generator and two-generator formalisms. In this paper we study the advantages and disadvantages of each, using classical problems with known solutions and synthetic data.

Figures (17)

• Figure 1: Configuration scheme of the Structure Preserving Neural Network (SPNN) employed to learn single generator (a) and GENERIC (b) formalisms. The input parameters of the net are the state of the system, $\boldsymbol{z}(\boldsymbol{x},t)$, at each time step. The output of the net includes the energy $\mathcal{F}$, $\mathcal{H}$, and $\mathcal{S}$, as well as the $\boldsymbol{m}$, and $\boldsymbol{l}$ components needed to reconstruct the formalism. The integration of the formalism gives the state of the system at the next time step $\boldsymbol{z}(\boldsymbol{x},t+1)$. Then, the Data error and the degeneracy conditions are computed to define the loss of the net.
• Figure 2: Double thermo-elastic pendulum system.
• Figure 3: Loss representation in training for GENERIC (a) and single generator (b) formalisms (Log scale). The loss function of the GENERIC includes data loss, $\mathcal{L}^{\text{data}}$, and degeneracy loss $\mathcal{L}^{\text{degen}}$, while the only contribution on the single generator is the data loss, $\mathcal{L}^{\text{data}}$.
• Figure 4: Results of the reconstruction of the double thermo-elastic pendulum system. Test trajectory (Ground Truth, GT) and the reconstruction of the double thermo-elastic pendulum using time integration with GENERIC formalism.
• Figure 5: Results of the reconstruction of the double thermo-elastic pendulum system. Test trajectory (Ground Truth, GT) and the reconstruction of the double thermo-elastic pendulum using time integration with single generator formalism.