Thermodynamic Regulation of Finite-Time Gibbs Training in Energy-Based Models: A Restricted Boltzmann Machine Study

TL;DR

An endogenous thermodynamic regulation framework in which temperature evolves as a dynamical state variable coupled to measurable sampling statistics is introduced, which establishes global parameter boundedness under strictly positive L2 regularization and proves local exponential stability of the thermodynamic subsystem.

Abstract

Restricted Boltzmann Machines (RBMs) are typically trained using finite-length Gibbs chains under a fixed sampling temperature. This practice implicitly assumes that the stochastic regime remains valid as the energy landscape evolves during learning. We argue that this assumption can become structurally fragile under finite-time training dynamics. This fragility arises because, in nonconvex energy-based models, fixed-temperature finite-time training can generate admissible trajectories with effective-field amplification and conductance collapse. As a result, the Gibbs sampler may asymptotically freeze, the negative phase may localize, and, without sufficiently strong regularization, parameters may exhibit deterministic linear drift. To address this instability, we introduce an endogenous thermodynamic regulation framework in which temperature evolves as a dynamical state variable coupled to measurable sampling statistics. Under standard local Lipschitz conditions and a two-time-scale separation regime, we establish global parameter boundedness under strictly positive L2 regularization. We further prove local exponential stability of the thermodynamic subsystem and show that the regulated regime mitigates inverse-temperature blow-up and freezing-induced degeneracy within a forward-invariant neighborhood. Experiments on MNIST demonstrate that the proposed self-regulated RBM substantially improves normalization stability and effective sample size relative to fixed-temperature baselines, while preserving reconstruction performance. Overall, the results reinterpret RBM training as a controlled non-equilibrium dynamical process rather than a static equilibrium approximation.

Figures (5)

• Figure 1: Adaptive temperature-control regime (Seed 1). Samples were obtained after 6000 steps of ensemble Gibbs sampling. The generated configurations organize along a coherent correlation manifold. Digits containing multiple interdependent stroke structures show clearer structural persistence, signifying improved modeling of higher-order associative patterns within the regulated stochastic regime.
• Figure 2: Fixed temperature setting ($T=1$, Seed 1). Samples were obtained after 6000 steps of ensemble Gibbs sampling. The distribution displays comparatively constrained generative diversity, with compressed correlation geometry and reduced variation within correlated stroke structures.
• Figure 3: Frozen temperature setting ($T = T^{\ast}$, Seed 1). Samples were obtained after 6000 steps of ensemble Gibbs sampling. The generative configurations display sensitivity to the initialization and to the selection of fixed temperature, reflecting unstable correlation alignment and reduced architectural robustness.
• Figure 4: Microscopic flip dynamics and temperature evolution under endogenous regulation, according to adaptive setting, Seed is 1 for this graph and Figure \ref{['fig:energy_dynamics']}. Top: flip rate with adaptive reference, illustrating progressive suppression of stochastic transitions. Middle: global temperature trajectory. Bottom: micro--macro temperature decomposition. The decay in flip activity reflects the controlled thermodynamic stabilization mechanism.
• Figure 5: Energy-based diagnostics during training. Top: weight norm evolution. Second: effective inverse temperature $\beta_{\mathrm{eff}} = \|W_t\|_{F} / T_t$, where $\|\cdot\|_{F}$ denotes the Frobenius norm, capturing the macroscopic scaling of the energy landscape relative to temperature. This quantity is distinct from the field-level inverse temperature $\beta^{\mathrm{field}}$ appearing in Theorem \ref{['thm:freezing']}. Third: free energy comparison between data and model. Bottom: spectral inverse temperature $\beta_{\mathrm{spectral}} = \|W_t\|_{2} / T_t$, where $\|\cdot\|_{2}$ denotes the spectral norm (largest singular value).

Theorems & Definitions (35)

• Claim 1: Absence of Structural Guarantee under Fixed Temperature
• Claim 2: Local Stability upon Endogenous Thermodynamic Regulation
• Lemma 1: Exponential bound for the logistic variance term
• proof
• Theorem 1: Asymptotic freezing under diverging effective inverse temperature
• proof
• Remark 1
• Lemma 2: Conductance Collapse under Diverging Effective Field
• proof
• Remark 2