Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Near-Critical Dynamics in Heavy-Tailed Neural Networks

Ryota Kojima

Abstract

The criticality hypothesis posits that biological neural networks operate near a phase transition, yet within standard Gaussian mean-field theories this regime appears fragile and requires fine tuning. Here we show that heavy-tailed synaptic connectivity provides a robust alternative mechanism. By developing a dynamical mean-field theory for Cauchy-distributed couplings, we reduce the macroscopic dynamics to a one-dimensional gradient flow with a global Lyapunov potential. The resulting theory exhibits a continuous phase transition in which collective activity grows with the square root of the distance to criticality, and static susceptibility diverges only as the square root rather than linearly as in Gaussian mean-field theories. This structure gives rise to an emergent automatic gain control: activity-dependent noise fluctuations suppress the effective gain at high activity levels while preserving high susceptibility near the critical point. Extending this mechanism to general symmetric $α$-stable inputs, we identify heavy-tailed synapses as a key microscopic origin of robust near-critical dynamics in disordered neural circuits.

Robust Near-Critical Dynamics in Heavy-Tailed Neural Networks

Abstract

The criticality hypothesis posits that biological neural networks operate near a phase transition, yet within standard Gaussian mean-field theories this regime appears fragile and requires fine tuning. Here we show that heavy-tailed synaptic connectivity provides a robust alternative mechanism. By developing a dynamical mean-field theory for Cauchy-distributed couplings, we reduce the macroscopic dynamics to a one-dimensional gradient flow with a global Lyapunov potential. The resulting theory exhibits a continuous phase transition in which collective activity grows with the square root of the distance to criticality, and static susceptibility diverges only as the square root rather than linearly as in Gaussian mean-field theories. This structure gives rise to an emergent automatic gain control: activity-dependent noise fluctuations suppress the effective gain at high activity levels while preserving high susceptibility near the critical point. Extending this mechanism to general symmetric -stable inputs, we identify heavy-tailed synapses as a key microscopic origin of robust near-critical dynamics in disordered neural circuits.
Paper Structure (22 sections, 135 equations, 3 figures)

This paper contains 22 sections, 135 equations, 3 figures.

Table of Contents

  1. Dimensionality Reduction for Cauchy Networks
  2. Characteristic Functional of the Filtered Input
  3. Commutativity of Path Averaging and Time Integration
  4. Internal loop gain for Heaviside thresholds
  5. Critical scaling of the susceptibility
  6. L$^{1}$-Norm Scaling of Filtered Cauchy Input and Non-Self-Averaging
  7. L$^{1}$ scaling for nonnegative causal test filters
  8. Implications for non-self-averaging
  9. Universality Beyond Heaviside: General Nonnegative Rate Functions
  10. Landau expansion and critical exponents
  11. Concrete examples
  12. Logistic sigmoid.
  13. Saturated ReLU.
  14. Effective and closed-loop sensitivities: general $A_{\mathrm{eff}}$ and saturation factor $K(\phi,m)$
  15. Analytical Derivation of Cancellation Coefficients
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Microscopic verification and continuous phase transition. (Left) Solid curves show the macroscopic noise scale $\sigma(t)$ measured from microscopic network simulations. Here $\sigma$ is defined as the interquartile range/2 of the filtered input across neurons. Dashed curves show the solutions of the reduced one-dimensional ODE \ref{['eq:exact_ODE']} initialized at the same $\sigma(0)$. (Center) The squared stationary noise scale $\sigma_{*}^2$ (blue circles) scales linearly with the coupling strength $g$ near the onset of activity, consistent with the mean-field Landau exponent $\beta=1/2$. E (Right) Critical decay dynamics at $g \approx g_c$. Both the microscopic noise scale $\sigma(t)$ (blue) and the population activity $m(t)$ (red) exhibit algebraic decay $\propto t^{-1/2}$ (green dashed guide), consistent with the Landau mean-field universality class.
  • Figure 2: (Left) Stability landscape. The curvature of the Lyapunov potential, $1 - \mathcal{L}_*$, is plotted against the stationary population rate $m_*$. The theoretical prediction (dashed line) is in agreement with numerical simulations (circles), indicating that the system remains in a stable regime over the explored activity range, while the curvature becomes small near criticality ($m_* \to 0$). (Center) Automatic Gain Control (AGC). The closed-loop susceptibility $d\sigma_*/dh$ scales inversely with activity. Simulation data (squares) follow the theoretical prediction (black dashed) and the asymptotic $1/m_*$ scaling (green dotted), illustrating the "divisive brake". (Right) Bounds for sign-changing kernels. We use a biphasic difference-of-exponentials kernel $G(t) = h_{\tau_1}(t) - b h_{\tau_2}(t)$ with $h_\tau(t)=\tau^{-1} e^{-t/\tau}\Theta(t)$, and various $b$.
  • Figure S1: (Left) Quantile–Quantile (Q–Q) plot of the effective input field $\tilde{x}(t)$ filtered by an exponential kernel $G(t) = \tau^{-1} e^{-t/\tau}\Theta(t)$ with $\tau=1$ (corresponding to $L=1$), against a standard Cauchy distribution. The data collapsing onto the diagonal $y=x$ confirms that the inputs follow the Cauchy statistics predicted by the mean-field theory. (Right) Validation of the $L^1$-norm scaling law using a family of exponential measurement filters $G_L(t) = (L/\tau) e^{-t/\tau}\Theta(t)$ with $\tau=1$ and $L\in\{0.5,1.0,1.5,2.0\}$, whose $L^1$ norm is $\|G_L\|_1=L$. The stationary noise scale $\sigma_*$ estimated from the filtered trajectories scales linearly with $L=\|G_L\|_1$, in agreement with the theoretical prediction $\sigma_* = g m_* L$. Error bars indicate the standard deviation across 20 disorder realizations.