Structure of activity in multiregion recurrent neural networks

TL;DR

Unlike previous models of routing in neural circuits, which suppressed the activities of neuronal groups to control signal flow, routing in this model is achieved by exciting different high-dimensional activity patterns through a combination of connectivity structure and nonlinear recurrent dynamics.

Abstract

Neural circuits comprise multiple interconnected regions, each with complex dynamics. The interplay between local and global activity is thought to underlie computational flexibility, yet the structure of multiregion neural activity and its origins in synaptic connectivity remain poorly understood. We investigate recurrent neural networks with multiple regions, each containing neurons with random and structured connections. Inspired by experimental evidence of communication subspaces, we use low-rank connectivity between regions to enable selective activity routing. These networks exhibit high-dimensional fluctuations within regions and low-dimensional signal transmission between them. Using dynamical mean-field theory, with cross-region currents as order parameters, we show that regions act as both generators and transmitters of activity -- roles that are often in tension. Taming within-region activity can be crucial for effective signal routing. Unlike previous models that suppressed neural activity to control signal flow, our model achieves routing by exciting different high-dimensional activity patterns through connectivity structure and nonlinear dynamics. Our analysis offers insights into multiregion neural data and trained neural networks.

Figures (11)

• Figure 1: (a) Top: Schematic of the synaptic connectivity model. Different regions, each with "random plus rank-one" connectivity, are linked via rank-one matrices representing communication subspaces. In this network of $R=4$ regions, we highlight the rank-one and disordered couplings in region $\mu$, as well as the structured couplings to and from region $\nu$. Rank-one connections are defined through the outer product of vectors $\bm{m}^{\mu \nu}$ and $\bm{n}^{\mu\nu}$. Bottom: Tensor $T^{\mu\nu\rho}$, which encodes the geometric arrangement of the connectivity patterns and determines the dynamics of region-to-region currents in the mean-field picture. (b) Anatomical bottleneck or effective bottleneck implementing a rank-one connectivity matrix between regions $\nu$ and $\mu$. The dashed circle represents a linear neuron with fast timescale.
• Figure 2: (a) Restriction to the effective-interaction tensor $T^{\mu\nu\rho}$ corresponding to enforcing symmetry. This constraint sets $T^{\mu\nu\rho}=\delta^{\mu\rho} c^{\mu\nu}$, where $c^{\mu\nu}$ is a symmetric matrix. nonzero overlaps between connectivity patterns are indicated by colored auras, with equal colors indicating equal overlaps. In this scenario with $R = 4$ regions, the connectivity has 10 independent parameters: 4 for direct and 6 for indirect effective self-interactions. (b) Illustration of subspace-based routing in the case of symmetric effective interactions. When the activity subspace defined by the span of ${m}_i^{\mu \mu}$ in region $\mu$ is excited, bidirectional communication between regions $\mu$ and $\nu$ is suppressed, and vice versa, due to the nonlinear dynamics of the network.
• Figure 3: Structure of fixed points in networks with symmetric effective interactions. The same information for three different cases is shown on the left, center and right. (a) Values of $a^\mu$ and $b^\mu$ in the $R=5$ regions. (b) Dynamics of sampled neurons (left) and of incoming currents (right) in large simulations for each region. (c) Visualization of the steady-state current matrix $S^{\mu \nu}_0$ (left) and of the $L^2$-norms of the rows of this matrix (right). We show row-norms from the simulations (red dots) alongside analytical predictions (blue dot). In the leftmost plots, all regions are in non-routing mode. In the middle plots, region 1 is in non-routing mode and regions 2--4 are in routing mode. In the rightmost plots, regions 1 and 2 are in non-routing mode and regions 3--5 are in routing mode.
• Figure 4: Structure of activity in networks with disorder and symmetric effective interactions among regions. (a) Relationship between $A^\mu$ and $g^\mu$ for various values of $b^\mu$ in the DMFT. Dashed lines indicate nonphysical solutions of the DMFT equations corresponding to unstable fixed points. (b) Solutions for the two-point function $\Delta^\mu(\tau)$ for the parameter values indicated by the markers in (a). (c--e) are the same as (a--c) in Fig. \ref{['fig:symmetric']}, but with disorder, whose levels are shown in (a). All regions have $g^\mu > 1$, so regions produce high-dimensional fluctuations unless tamed by current-based activity. In the leftmost plots, chaos is suppressed in all regions, and all regions are in routing mode. In the middle plots, all regions are in routing mode, and high-dimensional fluctuations exist alongside the structured current-based activity in region 1. In the rightmost plots, region 1 is in disorder-dominated non-routing mode, and regions 2--5 are in routing mode. In chaotic regimes (middle and right columns), the inter-region currents converge to steady values despite ongoing chaotic dynamics. This convergence occurs because the readout patterns project out the chaotic fluctuations, though small $\mathcal{O}(1/\sqrt{N})$ fluctuations remain around the mean-field values.
• Figure 5: Dynamic behaviors in networks with asymmetric effective interactions ($R=2$ regions). (a) Most common dynamic behavior across 50 realizations of $T^{\mu\nu\rho}$, as a function of the leading eigenvalue $\lambda$ of $\hat{T}^{\mu\nu,\rho\sigma}$. (b) Entropy of the distribution over dynamic behaviors at each $\lambda$. (c) Example time series of currents $S^{\mu \nu}$ (top) and two-point functions $\hat{\Delta}^\mu(\tau)$ (bottom) for each dynamic behavior. In the top row, colors represent different currents; in the bottom row, black and gray lines represent the two regions.