Random matrix theory of sparse neuronal networks with heterogeneous timescales

TL;DR

The paper develops a sparse non-Hermitian random-matrix theory to explain how trained excitatory/inhibitory networks for working memory organize near-marginal discrete attractors. By modeling the Jacobian with an inhibitory-core excitatory-periphery motif and heterogenous timescales, it derives a SUSY-based framework that yields a spectral-edge condition linking sparsity, weight variances, E/I balance, and distributions of timescales and gains. The key finding is that slow, distributed inhibitory dynamics push the spectrum toward the instability boundary, producing near-marginal modes that enable robust input-driven transitions, while sparsity and non-normality shape the bar-and-blob spectral geometry. The approach unifies a detailed empirical picture of trained networks with analytic predictions, recovers known dense limits, and provides a tractable path to quantify stability landscapes in high-dimensional, structured neural systems.

Abstract

Training recurrent neuronal networks consisting of excitatory (E) and inhibitory (I) units with additive noise for working memory computation slows and diversifies inhibitory timescales, leading to improved task performance that is attributed to emergent marginally stable equilibria [PNAS 122 (2025) e2316745122]. Yet the link between trained network characteristics and their roles in shaping desirable dynamical landscapes remains unexplored. Here, we investigate the Jacobian matrices describing the dynamics near these equilibria and show that they are sparse, non-Hermitian rectangular-block matrices modified by heterogeneous synaptic decay timescales and activation-function gains. We specify a random matrix ensemble that faithfully captures the spectra of trained Jacobian matrices, arising from the inhibitory core - excitatory periphery network motif (pruned E weights, broadly distributed I weights) observed post-training. An analytic theory of this ensemble is developed using statistical field theory methods: a Hermitized resolvent representation of the spectral density processed with a supersymmetry-based treatment in the style of Fyodorov and Mirlin. In this manner, an analytic description of the spectral edge is obtained, relating statistical parameters of the Jacobians (sparsity, weight variances, E/I ratio, and the distributions of timescales and gains) to near-critical features of the equilibria essential for robust working memory computation.

Figures (9)

• Figure 1: Input-driven transitions between discrete attractors as a paradigm of working memory computation (a delayed match-to-sample task). A recurrent E/I rate network governed by Eq. \ref{['eq:dyn_model']} receives binary cues $s_1, s_2 \in \{\pm1\}$ through $W^{\mathrm{in}}$ and independent noise via $W^{\mathrm{noise}}$ (bottom left). The trial timeline (top) consists of cue $s_1$, a delay, cue $s_2$, and a response window. In the network state space (bottom), $s_1$ selects one of two delay-period operating fixed points $\mathbf{x}^*(\pm1)$; the second cue then drives a controlled hop to one of four discrete attractors $\mathbf{x}^*(s_1,s_2)$. A linear readout (not shown) reports the XNOR decision (match vs. non-match) from the firing pattern of the final attractor, $\sigma(\mathbf{x}^*(s_1,s_2))$. Reliable working-memory performance in this dynamical landscape requires these operating points to be near-marginal, stable enough during the delay to encode input history yet easily driven by the next input to hop to the subsequent operating point's basin of attraction. The emergent network motifs underlying near-marginal dynamical landscapes and their typical Jacobian spectra evaluated around operating points are shown in Fig. \ref{['fig:motif_spectra']}.
• Figure 2: Emergent inhibitory core---excitatory periphery motif and typical Jacobian spectra after training with noise. Starting from a dense Gaussian E/I network, training on the DMS task with additive noise, as in Eq. \ref{['eq:dyn_model']}, yields an inhibitory core---excitatory periphery architecture (top): excitatory outputs are pruned while inhibition dominates but relatively sparse. (Bottom): typical eigenvalue spectra of the Jacobian at a delay-period operating point $\mathbf{x}^*(s_1)$ of Fig. \ref{['fig:wm_sketch']}. This Jacobian is characterized by the random matrix ensemble $J(\mathbf{x}^*)=\mathcal{T}^{-1}\left(-I+W\mathcal{H}^*\right)$, Eqs. (\ref{['eq:Jacobian_operating_point']}-\ref{['eq:J_component']}). With weak-noise training (bottom left), $\tau_I \sim \tau_E$ and the bulk remains far from the edge of instability (vertical dashed line). With moderate-noise training, inhibitory timescales separate by roughly an order of magnitude, producing a characteristic bar-and-blob spectral geometry. As we will show, a real-axis bar is trivially set by excitatory timescales while slow inhibitory timescales push the inhibitory blob toward the edge of instability, i.e., near-marginal operating points (bottom right). In Sec. \ref{['sec: setup_RMT']} we formalize these post-training statistics with a sparse non-Hermitian E/I ensemble; Sec. \ref{['sec:jacobian-SUSY']} develops an analytic random matrix theory that predicts the bar-and-blob shape and a spectral-edge condition linking sparsity, E/I ratio, and the distributions of $\tau$ and $h$ to the blob’s proximity to $\Re\lambda=0$.
• Figure 3: Typical trained synaptic weights. The emergent $W_{ij}$ trained without noise $C=0$ (a) and with moderate noise $C=10$ (b) both exhibit inhibitory core---excitatory periphery motifs; outbound excitation is largely suppressed while inhibition dominates. The outgoing inhibitory weights are also quite sparse, see Fig. \ref{['fig:fitted_weight']} (bottom). Here $N = 200$. The displayed weights are representative samples obtained from the public dataset accompanying Ref. Rungratsameetaweemana2025 (https://osf.io/dqy3g/).
• Figure 4: Statistics of the trained inhibitory core. Here, we characterize the distribution of the non-zero inhibitory weights $W_{\bullet I}$ and their connectivity patterns, justifying the sparse non-Hermitian ensemble assumptions. (a)-(b) The distribution of non-zero inhibitory weights fits a Gaussian distribution. (c)-(f) The degree distributions (number of non-zero connections per neuron) for inhibitory-to-inhibitory ($I \to I$) and inhibitory-to-excitatory ($I \to E$) projections are well-captured by Binomial distributions. This empirically justifies modeling the connectivity matrix as a sparse Erdős-Rényi graph with finite mean degree $k$, as defined in Eq. \ref{['eq:sparse-ensemble']}. Statistics are derived from representative samples in the public dataset accompanying Ref. Rungratsameetaweemana2025 (https://osf.io/dqy3g/).
• Figure 5: Trained heterogeneous synaptic timescales and gain function distributions. Histograms show the empirical statistics of synaptic decay timescales $\tau$ (top row) and activation gains $h = \sigma'(x^*) \in [0,1/4]$ (bottom row) for excitatory (red) and inhibitory (blue) populations, aggregated across all trained networks (50 samples). Solid lines are Beta distribution fits. The biological synaptic timescales from Rungratsameetaweemana2025 range from 25 to 125 ms. (a) In networks trained without noise ($C=0$), both populations exhibit similar, fast timescale distributions ($\tau_E \sim \tau_I$). Fitted Beta parameters $(\alpha, \beta)$ are: Excitatory $(0.274, 1.66)$; Inhibitory $(0.273, 0.598)$. (b) Training with moderate noise ($C=10$) induces a separation of timescales where inhibitory units develop significantly slower dynamics ($\tau_E \ll \tau_I$). Fitted parameters: Excitatory $(0.35, 1.1)$; Inhibitory $(0.973, 0.473)$. (c)-(d) The gain distributions remain sharply peaked near zero for both noise conditions. (c) ($C=0$) Fitted parameters: Excitatory $(0.306, 6.51)$; Inhibitory $(0.496, 0.656)$. (d) ($C=10$) Fitted parameters: Excitatory $(0.254, 6.88)$; Inhibitory $(0.565, 0.967)$.