Continuous-Time Homeostatic Dynamics for Reentrant Inference Models

TL;DR

The paper develops a continuous-time neural-ODE formulation of the Fast-Weights Homeostatic Reentry Network (FHRN), unifying fast associative memory, a state-dependent reentry operator, and radial homeostasis into a two-timescale dynamical system. Through Jacobian spectral analysis and a Lyapunov-based argument, it proves bounded attractors on a ring manifold and input-to-state stability under explicit conditions, identifying a reflective regime where reentry yields stable oscillatory trajectories rather than runaway dynamics. Numerical investigations reveal how the reentry operator’s antisymmetric and symmetric components shape phase-space flows, establishing stable regions and illustrating the trade-off between reentry gain and homeostatic normalization. The work positions FHRN as a distinct computational class that enables self-referential, bounded internal inference, bridging biologically inspired gain control with continuous neural dynamics and reflective computation. This framework offers a principled path toward architectures capable of working memory, iterative internal processing, and reflection-like dynamics in a stable, analytically tractable setting.

Abstract

We formulate the Fast-Weights Homeostatic Reentry Network (FHRN) as a continuous-time neural-ODE system, revealing its role as a norm-regulated reentrant dynamical process. Starting from the discrete reentry rule $x_t = x_t^{(\mathrm{ex})} + γ\, W_r\, g(\|y_{t-1}\|)\, y_{t-1}$, we derive the coupled system $\dot{y}=-y+f(W_ry;\,x,\,A)+g_{\mathrm{h}}(y)$ showing that the network couples fast associative memory with global radial homeostasis. The dynamics admit bounded attractors governed by an energy functional, yielding a ring-like manifold. A Jacobian spectral analysis identifies a \emph{reflective regime} in which reentry induces stable oscillatory trajectories rather than divergence or collapse. Unlike continuous-time recurrent neural networks or liquid neural networks, FHRN achieves stability through population-level gain modulation rather than fixed recurrence or neuron-local time adaptation. These results establish the reentry network as a distinct class of self-referential neural dynamics supporting recursive yet bounded computation.

Figures (4)

• Figure 1: Computational architecture of the Fast-Weights Homeostatic Reentry Network. At each timestep, the model receives an external token embedding $x_t^{(\mathrm{ex})}$ and combines it with a reentrant signal $\gamma W_r g(\lVert y_{t-1}\rVert)\,y_{t-1}$ to form the effective block input $x_t$. The queries and keys are derived via learned projections. The fast associative memory $A_t = U_t^\top V_t$ stores low-rank temporal correlations and modulates the internal representation $y_t$ on subsequent steps. The reentrant loop provides recurrent reflective influence regulated by the homeostatic gain $g(\lVert y_t\rVert)$.
• Figure 2: Phase–space behavior of the continuous FHRN dynamics under different forms of the reentrant operator. (a) A purely antisymmetric operator yields rotational flow and a stable ring attractor. (b) Local Jacobian spectral map showing the zero-real-eigenvalue contour aligning with the attractor radius. (c) A symmetric operator produces saddle-like expansion and contraction without rotational structure. (d) A mixed operator generates hybrid spiral dynamics, combining rotation and anisotropic gain.
• Figure 3: Stability of the continuous-time FHRN dynamics under the homeostatic field. (a) Lyapunov energy landscape of the dynamics defined by $V(y) = \tfrac{1}{4}(\|y\|^2-1)^2$. The system monotonically descends this energy surface toward the minimum manifold. (b) Radial evolution $\lVert y(t)\rVert$ for different initial conditions, demonstrating monotonic regulation toward the homeostatic manifold.
• Figure 4: Stability structure of FHRN dynamics across the parameter space. (a) Critical stability surface showing the boundary $\|W_r\|_{\mathrm{crit}} = 1 / (\gamma\, g(\|y\|))$ across the $(\gamma,\beta)$ parameter space. Regions below the surface correspond to stable reentrant dynamics, while values above the boundary predict instability. (b) Two–dimensional stability map for fixed radius $\|y\|=1.1$, illustrating how stability varies with homeostasis $\beta$ and reentry magnitude $\|W_r\|$. Blue regions indicate contractive dynamics, while the transition curve ($r_e=1$) marks the onset of oscillatory or weakly divergent behavior.