Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The determining role of covariances in large networks of stochastic neurons

Vincent Painchaud, Patrick Desrosiers, Nicolas Doyon

TL;DR

The paper tackles the challenge of modeling large, stochastic neural networks where mean-field approaches fail to capture correlations and refractoriness. It constructs a microscopic continuous-time Markov chain over neuron states and derives a macroscopic, non-autonomous system for population means; to close the moment equations, it advances from a naive closure to a robust second-order closure using a sigmoid-expectation approximation. The resulting closed system includes covariances between active and refractory fractions, can alter fixed points and oscillatory behavior relative to mean-field predictions, and shows closer agreement with stochastic simulations across multiple scenarios. This work provides a practical, low-dimensional framework for understanding correlated neuronal activity in large networks and highlights the essential role of higher moments in neural dynamics.

Abstract

Biological neural networks are notoriously hard to model due to their stochastic behavior and high dimensionality. We tackle this problem by constructing a dynamical model of both the expectations and covariances of the fractions of active and refractory neurons in the network's populations. We do so by describing the evolution of the states of individual neurons with a continuous-time Markov chain, from which we formally derive a low-dimensional dynamical system. This is done by solving a moment closure problem in a way that is compatible with the nonlinearity and boundedness of the activation function. Our dynamical system captures the behavior of the high-dimensional stochastic model even in cases where the mean-field approximation fails to do so. Taking into account the second-order moments modifies the solutions that would be obtained with the mean-field approximation, and can lead to the appearance or disappearance of fixed points and limit cycles. We moreover perform numerical experiments where the mean-field approximation leads to periodically oscillating solutions, while the solutions of the second-order model can be interpreted as an average taken over many realizations of the stochastic model. Altogether, our results highlight the importance of including higher moments when studying stochastic networks and deepen our understanding of correlated neuronal activity.

The determining role of covariances in large networks of stochastic neurons

TL;DR

The paper tackles the challenge of modeling large, stochastic neural networks where mean-field approaches fail to capture correlations and refractoriness. It constructs a microscopic continuous-time Markov chain over neuron states and derives a macroscopic, non-autonomous system for population means; to close the moment equations, it advances from a naive closure to a robust second-order closure using a sigmoid-expectation approximation. The resulting closed system includes covariances between active and refractory fractions, can alter fixed points and oscillatory behavior relative to mean-field predictions, and shows closer agreement with stochastic simulations across multiple scenarios. This work provides a practical, low-dimensional framework for understanding correlated neuronal activity in large networks and highlights the essential role of higher moments in neural dynamics.

Abstract

Biological neural networks are notoriously hard to model due to their stochastic behavior and high dimensionality. We tackle this problem by constructing a dynamical model of both the expectations and covariances of the fractions of active and refractory neurons in the network's populations. We do so by describing the evolution of the states of individual neurons with a continuous-time Markov chain, from which we formally derive a low-dimensional dynamical system. This is done by solving a moment closure problem in a way that is compatible with the nonlinearity and boundedness of the activation function. Our dynamical system captures the behavior of the high-dimensional stochastic model even in cases where the mean-field approximation fails to do so. Taking into account the second-order moments modifies the solutions that would be obtained with the mean-field approximation, and can lead to the appearance or disappearance of fixed points and limit cycles. We moreover perform numerical experiments where the mean-field approximation leads to periodically oscillating solutions, while the solutions of the second-order model can be interpreted as an average taken over many realizations of the stochastic model. Altogether, our results highlight the importance of including higher moments when studying stochastic networks and deepen our understanding of correlated neuronal activity.
Paper Structure (26 sections, 7 theorems, 92 equations, 9 figures)

This paper contains 26 sections, 7 theorems, 92 equations, 9 figures.

Table of Contents

  1. Introduction
  2. The model
  3. A modelization of a biological neural network
  4. A description of the evolution of the network's state
  5. A macroscopic approximation of the Markov chain's behavior
  6. A second-order solution to the moment closure problem
  7. The naive approach and the need to go further
  8. Approximation of a sigmoid function's expectation
  9. The moment closure
  10. The dynamical system
  11. Examples
  12. A word on the methodology
  13. Covariances can improve the model's accuracy
  14. The second-order model can describe the averaging of multiple steady states
  15. The second-order model can describe the averaging of oscillations
  16. ...and 11 more sections

Key Result

Theorem 1

Suppose that the thresholds in population $J$ follow a unimodal and symmetric distribution with mean $\theta_J$ and cumulative distribution function $F_{\theta_J}$. Let $g_J$ and $G_J$ be defined by eq.defgJ.maineq.defGJ.main respectively. Suppose that Then $G_J$ satisfies the following conditions.

Figures (9)

  • Figure 1: Neurons' states and allowed transitions between them with corresponding rates. Here, $i$ denotes the imaginary unit.
  • Figure 2: Solution of the dynamical system from \ref{['eq.DSTaylor']} with exponentially diverging variances, an inadmissible behavior. The parameters and initial state were chosen as in the example of Section \ref{['sec.example1']}.
  • Figure 3: Example of comparison between $F_{\theta_J}(\IfNoValueTF{J}{\mathcal{B}}{\mathcal{B}_{J}})$, the actual expectation $\expect{F_{\theta_J}(B^J)}$, and our approximation $G_J(\IfNoValueTF{J}{\mathcal{B}}{\mathcal{B}_{J}}, \IfNoValueTF{JJ}{\mathrm{C}_{BB}}{\mathrm{C}_{BB}^{JJ}})$ for a fixed variance $\IfNoValueTF{JJ}{\mathrm{C}_{BB}}{\mathrm{C}_{BB}^{JJ}}$, assuming the thresholds follow a logistic distribution with mean $\theta_J$ and scaling factor $s_{\theta_J}$. To compute $\expect{F_{\theta_J}(B^J)}$, we assumed as a heuristic device that $B^J$ follows a logistic distribution with mean $\IfNoValueTF{J}{\mathcal{B}}{\mathcal{B}_{J}}$ and scaling factor $s_{\theta_J}$ as well, but this choice was made for illustration purposes only.
  • Figure 4: Numerical simulations for a network of a single population of 1000 neurons with parameters given in \ref{['eq.ex1.params']} and initial state given in \ref{['eq.ex1.initstate']} with $n = 1000$ distinct initial neuronal states. On the left panels is a comparison between the solution of the second-order dynamical system and statistics computed from 1000 simulated trajectories of the underlying Markov chain. The shaded regions around the curves are associated with statistics, and are bounded above and below by a difference of one standard deviation from the average value. On the right panel is a phase plane of the mean-field dynamical system, on which are plotted the solution of the mean-field system and the macroscopic behavior of a typical trajectory of the Markov chain. See Figure \ref{['fig.ex1.covs']} in Appendix \ref{['apx.covs']} for the other covariances.
  • Figure 5: Numerical simulations for a network consisting of a single population of 1000 neurons with parameters given in \ref{['eq.ex2.params']} and initial state given in \ref{['eq.ex2.initstate']} with $n = 100$ distinct initial neuronal states. On the left panels is a comparison between the solution of the second-order dynamical system and statistics computed from 1000 simulated trajectories of the underlying Markov chain. On the right panel is a phase plane of the mean-field dynamical system, on which are plotted the solution of the mean-field system and the macroscopic behaviors of two trajectories of the Markov chain. These trajectories are typical representatives of the sets of trajectories that converge to each fixed point. See Figure \ref{['fig.ex2.covs']} in Appendix \ref{['apx.covs']} for the other covariances.
  • ...and 4 more figures

Theorems & Definitions (11)

  • Theorem 1
  • Corollary 1
  • Corollary 2
  • Proposition 1
  • proof
  • Theorem 1
  • proof
  • Corollary 1
  • proof
  • Corollary 2
  • ...and 1 more