Formation of Artificial Neural Assemblies by Biologically Plausible Inhibition Mechanisms

Abstract

As proposed by Hebb's theory, neural assemblies are groups of excitatory neurons that fire synchronously and exhibit high synaptic density, representing external stimuli and supporting cognitive functions such as language and decision-making. Recently, a model called Assembly Calculus (AC) was proposed, enabling the formation of artificial neural assemblies through the $k$-winners-take-all selection process and Hebbian learning. Although the model is capable of forming assemblies according to Hebb's theory, the adopted selection process does not incorporate essential aspects of biological neural computation, as neural activity, which is often governed by statistical distributions consistent with power-law scaling. Given this limitation, the present work aimed to bring the model's dynamics closer to that observed in real cortical networks. To achieve this, a new selection mechanism inspired by the dynamics of gamma oscillation cycles, called E%-winners-take-all, was implemented, combined with an inhibition process based on the ratio between excitatory and inhibitory neurons observed in various regions of the cerebral cortex. The results obtained from our model (called E%-WTA model) were compared with those of the original model, and the analyses demonstrated that the introduced modifications allowed the network's own dynamics to determine the size of the formed assemblies. Furthermore, the recovery rate of these groups, through the evocation of the stimuli that generated them, became superior to that obtained in the original model.

Figures (4)

• Figure 1: Structure and dynamics of the E%-WTA model. a) Neurons in stimulus areas ($\mathcal{S}$) project only to neurons in memory areas ($\mathcal{M}$) (red arrow), whereas neurons in a memory area can form an internal recurrent network (green arrow), but cannot project back to the stimulus area. b) Neurons in memory areas can project to other memory areas, potentially forming a bidirectional flow of stimuli. c) The firing of a neuron in a memory area $\mathcal{M}$ is represented by the firing state function $(f)$, where $f(t)=1$ for a firing neuron (green), while $f(t)=0$ represents a non-firing neuron (gray). What determines the value of $f(t)$ is the E%-winners-take-all selection process, which depends on the neuron that received the greatest stimulation (yellow). In each iteration of the model, the neurons that fire can vary, depending on the dynamics of stimuli across the network. d) Synaptic weights are initialized according to Eq. \ref{['eq:weights']} and adjusted according to Hebb's rule. At the end of the formation process, a neural assembly will be created (green), exhibiting reinforced synapses (dark lines). e) Excitatory neurons are selected to fire during gamma oscillations. The most stimulated neuron fires first ($E_1$), opening a temporal window that allows other excitatory neurons to fire ($E_2$ and $E_3$) until it closes. When the temporal window ends, other excitatory neurons will not fire ($E_4$) because they received little stimulation and are therefore inhibited by inhibitory interneurons ($I$). f) In the original selection process, the number of neurons that fire is fixed, allowing neurons with weak stimulation to fire. In the approach proposed here, the E%-winners-take-all selection process allows a variable number of neurons to fire (red), but also prevents neurons that received little stimulation from firing (blue). Neurons that fire in each iteration meet the conditions of the E%-winners-take-all process (dashed line).
• Figure 2: Neural assembly formation in AC and E%-WTA models. a) Neural assemblies in the AC model form through the successive firing of a set of neurons in $\mathcal{S}$ (blue). Over iterations, new neurons fire for the first time (green), while other neurons might fire in multiple iterations (red). The condition for formation is that there are no new neurons ($|N_{10}| = 0$ ). b) AC model behavior during formation ($k_s = 37$). As iterations progress ($t$ - horizontal axis), the number of new neurons decreases ($|N_t|$ - left vertical axis, blue). For a certain value of $t$, $|N_t| = 0$ (vertical dashed line), indicating no more new neurons. When this occurs in the original model, a neural assembly is considered formed. However, if iterations continue, the neurons that fire won't always be those belonging to the formed assembly ($|X_t|$ - right vertical axis, red). Thus, even when the formation condition is satisfied, there is no guarantee that the assembly neurons will continue to fire synchronously as iterations progress. c) Behavior of the first two conditions in the E%-WTA model. In all simulations performed in this study, the first two conditions converged to zero. d) Failure rate (mean) of neural assembly formation in the E%-WTA model (normalized by the number of simulations for each parameter tested) as a function of synaptic plasticity ($\beta$) for different feedforward inhibition values ($\omega_{inh}$), considering a baseline size ($|A|_{min}$) of 6 neurons. After 100 simulations for each parameter ($\beta$ and $\omega_{inh}$), the best values for neural assembly formation in your model are: $\omega_{inh} = -0.2$ and $\beta \leq 0.01$. (Parameters: see Table \ref{['tab:tb_2']})
• Figure 3: Characteristics of the assemblies formed in the AC and E%-WTA models as a function of synaptic plasticity. a) Size of the neural assemblies in the E%-WTA model with and without (only positive synaptic weights) feedforward inhibition. b) Retrieval of the formed assemblies, where recovered portion ($|A|_{rec}$) was normalized by the size ($|A|$). c) Overlap matrix of the stimuli (right) and of the assemblies in a memory area (left). E%-WTA model – Top matrices; AC model – Bottom matrices. The diagonal of the matrices represents the size of the neural assemblies and the size of the stimuli. d) Distribution of the overlap ($\#|A_i\cap A_j|$) between the models ($\beta = 0.01$ and $k_s = 200$ for both models). Crosses ($\times$) represent the outliers of the presented distributions.
• Figure 4: E%-WTA model behavior as a function of a) the probability of synaptic connection ($p_s$) and b) the stimulus size ($k_s$) (200 simulations for each value of $p_s$ and $k_s$ with $\beta = 0.01$). Lower values lead to lower values of success rate, thereby limiting the applicability of operations defined in the original model. The success rate (normalized by the number of simulations) is defined here as the ratio between the number of simulations that formed a neural assembly and the total number of simulations. c) The formation of more than one neural assembly in a memory area allows only partial retrieval of the first neural assembly formed (50 simulations performed in each case with $\beta = 0.01$).