Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modeling and Contractivity of Neural-Synaptic Networks with Hebbian Learning

Veronica Centorrino, Francesco Bullo, Giovanni Russo

TL;DR

This paper is concerned with the modeling and analysis of two of the most commonly used recurrent neural network models (i.e., Hopfield neural network and firing-rate neural network) with dynamic recurrent connections undergoing Hebbian learning rules with biologically meaningful quantities.

Abstract

This paper is concerned with the modeling and analysis of two of the most commonly used recurrent neural network models (i.e., Hopfield neural network and firing-rate neural network) with dynamic recurrent connections undergoing Hebbian learning rules. To capture the synaptic sparsity of neural circuits we propose a low dimensional formulation. We then characterize certain key dynamical properties. First, we give biologically-inspired forward invariance results. Then, we give sufficient conditions for the non-Euclidean contractivity of the models. Our contraction analysis leads to stability and robustness of time-varying trajectories -- for networks with both excitatory and inhibitory synapses governed by both Hebbian and anti-Hebbian rules. For each model, we propose a contractivity test based upon biologically meaningful quantities, e.g., neural and synaptic decay rate, maximum in-degree, and the maximum synaptic strength. Then, we show that the models satisfy Dale's Principle. Finally, we illustrate the effectiveness of our results via a numerical example.

Modeling and Contractivity of Neural-Synaptic Networks with Hebbian Learning

TL;DR

This paper is concerned with the modeling and analysis of two of the most commonly used recurrent neural network models (i.e., Hopfield neural network and firing-rate neural network) with dynamic recurrent connections undergoing Hebbian learning rules with biologically meaningful quantities.

Abstract

This paper is concerned with the modeling and analysis of two of the most commonly used recurrent neural network models (i.e., Hopfield neural network and firing-rate neural network) with dynamic recurrent connections undergoing Hebbian learning rules. To capture the synaptic sparsity of neural circuits we propose a low dimensional formulation. We then characterize certain key dynamical properties. First, we give biologically-inspired forward invariance results. Then, we give sufficient conditions for the non-Euclidean contractivity of the models. Our contraction analysis leads to stability and robustness of time-varying trajectories -- for networks with both excitatory and inhibitory synapses governed by both Hebbian and anti-Hebbian rules. For each model, we propose a contractivity test based upon biologically meaningful quantities, e.g., neural and synaptic decay rate, maximum in-degree, and the maximum synaptic strength. Then, we show that the models satisfy Dale's Principle. Finally, we illustrate the effectiveness of our results via a numerical example.
Paper Structure (37 sections, 13 theorems, 76 equations, 4 figures)

This paper contains 37 sections, 13 theorems, 76 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Contributions
  4. Mathematical Background
  5. Contraction Theory
  6. Composite Norms
  7. Out-incidence and In-incidence Matrices
  8. Modeling
  9. Neural Dynamics
  10. Hopfield Neural Network
  11. Firing-rate Neural Network
  12. Synaptic Dynamics
  13. Coupled Neural Synaptic Models
  14. Hopfield-Hebbian Model
  15. Firing-rate-Hebbian Model
  16. ...and 22 more sections

Key Result

Lemma 3

Consider a Metzler matrix $M \in \mathbb{R}^{n\times n}$. For any $p \in [1, \infty]$, and $\delta > 0$, define $\eta_{M,p,\delta}\in \mathbb{R}^n_{\geq0}$ by where $q \in [1, \infty]$ is the conjugate index of $p$, while $l$ and $r \in \mathbb{R}^n_{\geq0}$ are the left and right dominant eigenvectors of $M+\delta \hbox{\fontencoding{U}1}_n \hbox{\fontencoding{U}1}_n^{\textsf{T}}$. Then for each

Figures (4)

  • Figure 1: Coupled neural-synaptic model with six neurons $i$ and six edges, $e_i$, $i=1,\dots,6$, four excitatory (green) and two inhibitory (red). Only nodes $1$ and $2$ are subjected to the external stimuli $u_1$ and $u_2$, respectively. Colors online.
  • Figure 2: Simulation of the Hopfield-Hebbian model of Figure \ref{['fig:num_example']} exhibiting entrainment to periodic inputs typical of contracting systems. See Section \ref{['Numerical Example']} for the parameters.
  • Figure 3: Coupled neural-synaptic model with six neurons $i$, $i=1,\dots,6$, and nine edges, $e_j$, $j=1,\dots,9$, six excitatory (green) and three inhibitory (red). Only nodes $1$, $2$, and $4$ are subjected to the external stimuli $u_1$, $u_2$, and $u_3$, respectively. Colors online.
  • Figure 4: Simulation of the Hopfield-Hebbian model of Figure \ref{['fig:recurrent_network']} with the activation functions affected by $2s$ of delay. Even with the delays the system appears to be still contracting. See Section \ref{['Numerical Example']} for the parameters.

Theorems & Definitions (34)

  • Definition 1
  • Definition 2
  • Lemma 3
  • Theorem 4
  • Proposition 5
  • Remark 1
  • Remark 2
  • Definition 6
  • Remark 3
  • Remark 4
  • ...and 24 more