Dissipative Learning: A Framework for Viable Adaptive Systems

TL;DR

The paper reframes learning as a dissipative process and introduces the Bayesian Emergent Dissipative Structures (BEDS) framework to model adaptive belief states under energy dissipation. It proves a Conditional Optimality Theorem: under three explicit assumptions (intrinsic information measure, maximum-entropy belief states, and quasi-static dissipation), Fisher–Rao regularization is the unique thermodynamically optimal regularization, with Euclidean regularization proven structurally suboptimal. The framework unifies a wide range of methods (Ridge, SIGReg, EMA, SAC, Transformers, diffusion models) under a single fundamental equation that combines Fisher–Rao distance with a data-fitting term. It further develops a three-sentence BEDS summary, a six-class problem taxonomy, and extensions to hierarchical and multi-agent systems, providing a principled lens on forgetting, crystallization, stability, and efficiency. The work outlines practical diagnostics, thermodynamic efficiency considerations, and testable predictions, positioning viability under finite resources as central to continual and distributed learning rather than asymptotic optimality.

Abstract

We propose a perspective in which learning is an intrinsically dissipative process. Forgetting and regularization are not heuristic add-ons but structural requirements for adaptive systems. Drawing on information theory, thermodynamics, and information geometry, we introduce the BEDS (Bayesian Emergent Dissipative Structures) framework, modeling learning as the evolution of compressed belief states under dissipation constraints. A central contribution is the Conditional Optimality Theorem, showing that Fisher-Rao regularization measuring change via information divergence rather than Euclidean distance is the unique thermodynamically optimal regularization strategy, achieving minimal dissipation. Euclidean regularization is shown to be structurally suboptimal. The framework unifies existing methods (Ridge, SIGReg, EMA, SAC) as special cases of a single governing equation. Within this view, overfitting corresponds to over-crystallization, while catastrophic forgetting reflects insufficient dissipation control. The framework distinguishes BEDS-crystallizable problems, where beliefs converge to stable equilibria, from BEDS-maintainable problems, which require continual adaptation. It extends naturally to continual and multi-agent systems, where viability, stability under adaptation and finite resources replaces asymptotic optimality as the primary criterion. Overall, this work reframes learning as maintaining viable belief states under dissipation constraints, providing a principled lens on forgetting, regularization, and stability.

Figures (14)

• Figure 1: Two paradigms for digital twins and their computational models. Left, a frozen snapshot trained at T0. As reality evolves, the model reality gap grows unboundedly, requiring periodic retraining. The underlying computational model is a Turing machine with unbounded tape: memory accumulates without limit, and the energy cost of maintaining fidelity diverges. Right, a continuously synchronized system that receives observations, updates beliefs, and exports entropy through dissipation. The gap remains bounded through perpetual synchronization. The computational model has bounded state: forgetting prevents memory explosion. This contrast, unbounded tape vs. bounded dissipative state, captures why classical models require ever-growing resources while BEDS systems remain viable indefinitely. The BEDS parameters $(\mu, \tau)$ shown are formally introduced in Definition \ref{['def:beds-state']}.
• Figure 2: Bénard cells, LeJEPA, and DINO---three dissipative structures. Left: In Bénard convection, hexagonal cells emerge from heat flux. Center: LeJEPA learns latten ($\mu$, $\tau$) through masked prediction and SIGReg entropy export. Right: DINO learns global coherence ($\phi$, $\kappa$) through student-teacher synchronization and centering. Together, LeJEPA and DINO span the full BEDS state space. Under A1--A3, these systems satisfy the conditions for thermodynamically optimal learning.
• Figure 3: The BEDS state space with the four canonical parameters.Top hemisphere: Learning trajectories illustrated by two complementary paradigms---LeJEPA learns on $(\mu, \tau)$ with vertical movement toward poles (increasing precision $\tau \to \infty$), while DINO learns on $(\varphi, \kappa)$ with spiral movement toward the surface (coherence $\kappa$ increases toward crystallization while phase $\varphi$ rotates). Systems may also experience forgetting (decreasing $\kappa$ and $\tau$) or collapse toward center ($\kappa, \tau \to 0$). Bottom hemisphere: The four BEDS parameters visualized---position $\mu$ (horizontal), precision $\tau$ (vertical), phase $\varphi$ (azimuthal rotation), and coherence $\kappa$ (radial distance). Poles represent high precision ($\tau \to \infty$); the surface represents crystallized states with maximum coherence ($\kappa_{\max}$); the center represents collapse ($\kappa \to 0$).
• Figure 4: From entropy to BEDS: The logical path. Each step follows from the previous, with assumptions A1, A2, A3 entering at specific points. Entropy equals uncertainty (Shannon). Order requires entropy export (Prigogine). A2 (MaxEnt) yields Gaussians and von Mises. A1 (Čencov) yields Fisher--Rao geometry. A3 (quasi-static) makes geodesics optimal. The result: BEDS state space $\mathbb{H}^2 \times \mathcal{M}_{\text{vM}}$.
• Figure 5: Recursive structure in deep networks. Each layer's posterior becomes the next layer's prior. Early layers learn low-level features (edges, textures) which "crystallize" into axioms for later layers. Under A2, this hierarchical belief propagation is well-defined through maximum-entropy distributions.

Theorems & Definitions (38)

• Remark 1.1: Landauer vs Computational Cost
• Definition 6.1: Fisher Information Matrix
• Proposition 6.1: Čencov's Theorem, 1982
• Proposition 6.2: Local KL-Fisher Correspondence
• Theorem 6.3: Conditional Optimality of Fisher--Rao Regularization
• proof
• Corollary 6.4: Energy-Precision Bound
• proof
• Corollary 6.5: Euclidean Regularization is Suboptimal
• proof