Bridging statistical mechanics and thermodynamics away from equilibrium: a data-driven approach for learning internal variables and their dynamics

TL;DR

This work tackles the challenge of formulating a statistical-mechanics-grounded framework for thermodynamics with internal variables in non-equilibrium settings. It introduces IB-VONNs, combining the Information Bottleneck (IB) principle, conditional normalizing flows (CNFs), and Variational Onsager Neural Networks (VONNs) to automatically uncover internal variables and learn their thermodynamically consistent evolution from microscopic Langevin data. The approach is validated on two overdamped Langevin-based problems: a single particle in an optical trap (where distributions are Gaussian and analytical benchmarks exist) and a mass-spring chain with a double-well potential (where distributions are multimodal). Results show that the latent internal variables faithfully capture key microstate features and that the learned macro-dynamics respect Markovianity and thermodynamic constraints, enabling accurate macroscopic predictions and microstate reconstruction. This framework thus provides a data-driven bridge between microscopic statistics and macroscopic thermodynamics away from equilibrium, with potential applicability to complex materials and phase-transforming systems.

Abstract

Thermodynamics with internal variables is a common approach in continuum mechanics to model inelastic (i.e., non-equilibrium) material behavior. While this approach is computationally and theoretically attractive, it currently lacks a well-established statistical mechanics foundation. As a result, internal variables are typically chosen phenomenologically and lack a direct link to the underlying physics which hinders the predictability of the theory. To address these challenges, we propose a machine learning approach that is consistent with the principles of statistical mechanics and thermodynamics. The proposed approach leverages the following techniques (i) the information bottleneck (IB) method to ensure that the learned internal variables are functions of the microstates and are capable of capturing the salient feature of the microscopic distribution; (ii) conditional normalizing flows to represent arbitrary probability distributions of the microscopic states as functions of the state variables; and (iii) Variational Onsager Neural Networks (VONNs) to guarantee thermodynamic consistency and Markovianity of the learned evolution equations. The resulting framework, called IB-VONNs, is tested on two problems of colloidal systems, governed at the microscale by overdamped Langevin dynamics. The first one is a prototypical model for a colloidal particle in an optical trap, which can be solved analytically, and thus ideal to verify the framework. The second problem is a one-dimensional phase-transforming system, whose macroscopic description still lacks a statistical mechanics foundation under general conditions. The results in both cases indicate that the proposed machine learning strategy can indeed bridge statistical mechanics and thermodynamics with internal variables away from equilibrium.

Figures (21)

• Figure 1: Overview of the IB-VONNs framework. It is composed of an encoder-decoder architecture that relates the microscopic probability distribution with the state variables at all times (based on the Information Bottleneck principle), and Variational Onsager Neural Networks to discover the evolution equations of the macroscopic fields. The state variables are composed of the usual variables to describe the system in equilibrium $\chi$ and the internal variables $\alpha$ to be discovered by means of this framework.
• Figure 2: A schematic plot of the encoder architecture, used to learn the internal variables $\boldsymbol \alpha$. Such variables are expressed as $\boldsymbol{\alpha} = \rho \left(\frac{1}{n_r} \sum_{i=1}^{n_r} h(\boldsymbol{x}^i(t)) \right)$, with $h$ and $\rho$ modeled with neural networks, in order to ensure invariance with respect to the input order of the realizations. Here, $n_r$ denotes the number of realizations.
• Figure 3: (a) A schematic plot of the CNFs decoder architecture (with $d=4$). The parameters $\boldsymbol{a}$, $\boldsymbol{b}$ and $\boldsymbol{c}$ are functions of the state variables $\boldsymbol{z}(t)$, and they are modeled by neural networks. $\sigma(\cdot)$ and $\sigma^{-1}(\cdot)$ are the sigmoid and logit activation functions, respectively. (b) A schematic plot of the GMM decoder architecture used in Example 2. It is composed of two Gaussians and parameterized by the mean strain $\varepsilon$, the variances of the two Gaussians, $\sigma_l$ and $\sigma_r$, and the mixing coefficient $\pi$.
• Figure 4: A schematic plot of the IB-VONNs architecture. It can be understood as an IB structure to learn the internal variables of the system, augmented with VONNs to learn the evolution equations of the state variables $\boldsymbol{\alpha}$ and $\boldsymbol{\chi}$. The two autoencoders in the figure refer to one single encoder and decoder applied on data at different times.
• Figure 5: A prototypical model for a colloidal particle in an optical trap. The particle is connected to a fix point via a spring and pulled via a spring according to a pulling protocol $\lambda(t)$.