CHANI: Correlation-based Hawkes Aggregation of Neurons with bio-Inspiration

TL;DR

CHANI introduces a mathematically grounded, biologically inspired spiking neural network where neurons operate as Hawkes processes and weights adapt via local expert-aggregation rules. The work proves that, under CHANI-EWA assumptions and suitable regimes, hidden layers converge to encode feature correlations, enabling neuronal assemblies that can support multi-class representations and even shared activations across classes. It establishes average and asymptotic learning guarantees, along with VC-dimension bounds, and demonstrates empirical viability on simulated and handwritten-digit datasets. Overall, the paper provides theoretical guarantees for local learning in SNNs, links to attention-like mechanisms, and a pathway toward understanding formation of assemblies in concept representation with potential impacts on neuro-inspired AI design.

Abstract

The present work aims at proving mathematically that a neural network inspired by biology can learn a classification task thanks to local transformations only. In this purpose, we propose a spiking neural network named CHANI (Correlation-based Hawkes Aggregation of Neurons with bio-Inspiration), whose neurons activity is modeled by Hawkes processes. Synaptic weights are updated thanks to an expert aggregation algorithm, providing a local and simple learning rule. We were able to prove that our network can learn on average and asymptotically. Moreover, we demonstrated that it automatically produces neuronal assemblies in the sense that the network can encode several classes and that a same neuron in the intermediate layers might be activated by more than one class, and we provided numerical simulations on synthetic dataset. This theoretical approach contrasts with the traditional empirical validation of biologically inspired networks and paves the way for understanding how local learning rules enable neurons to form assemblies able to represent complex concepts.

Figures (4)

• Figure 1: Sketch of CHANI algorithm. A. CHANI’s training consists of the sequential presentation of objects, with possible redundancy: here, objects 1 and 5 (respectively 3 and 6) share the same nature. Each object $m = 1, \dots, M$ or $N$ (depending on whether hidden layers are being trained or not) is presented for a duration $T$. B. (Step 1) The network senses the presented object. (Step 2) Depending on the presence of a given feature $i$ (e.g., color) in the object, neuron $i \in I$ emits more spikes during the presentation time $T$ (i.e., more “1”s in the sequence of length $T$ that it produces). (Step 3) In the first hidden layer, each neuron $j_{\{i_1\} \cup \{i_2\}} \in J_1$ searches for correlations between features $i_1$ and $i_2$ in the presented objects and updates its synaptic weights $w_m^{i \to j}$ after processing the $m^{th}$ object. Ideally, at the end of training, it spikes only when the object contains both features $i_1$ and $i_2$. (Step 4) Neurons that do not detect sufficient correlation across all presented objects during the first training phase (presentation of $M$ objects) are pruned. The resulting layer is denoted $\hat{J}_1$. C. (Step 5) As the layers are sequentially trained, a neuron $j_S$ in layer $J_{l-1}$ learns to detect the presence of all features $i \in S \subset I$ in the presented object. Ideally, at the end of training, this neuron spikes only when the object exhibits all features in $S$. The dashed arrows represent composite connections across several layers. (Step 6) In layer $J_l$, a neuron $j_{S_1 \cup S_2}$ searches for correlations between neurons $j_{S_1}$ and $j_{S_2} \in \hat{J}_{l-1}$. Ideally, after learning, it spikes only when the presented object contains all features in $S_1 \cup S_2$. All layers are successively pruned to retain only the relevant correlations in the presented objects. D. (Step 7) Once all hidden layers have been trained, the final layer $K$ is added. Each neuron $k \in K$ identifies the neurons $j_S \in \hat{J}_L$ that are the most sensible to objects of class $k$. At the end of training, ideally, neuron $k$ is connected to all $j_S \in \hat{J}_L$ such that the feature set $S$ is relevant for defining class $k$.
• Figure 2: The three sorts of discrepancies. Av. stands for average.
• Figure 3: Accuracy with confidence interval of level $0.9$ on simulated dataset for CHANI with EWA and CHANI with PWA, with zero and one hidden layer. The parameters are $T=2000$, $M=40$, $N = 360$, $p=0.5$ and each configuration was run $100$ times. For CHANI with EWA and zero hidden layer: $\eta^1 = 0.05$. For CHANI with EWA and one hidden layer: $\eta^1 = 3$ and $\eta^2 = 0.05$. For CHANI with PWA: $b=2$. For CHANI with EWA and PWA with one hidden layer: $s_1 = 0.1$. For each configuration, the training data are organized into epochs, where one epoch corresponds to a sequence of the $9$ object natures presented in random order. After each epoch of the output layer training, the network’s accuracy is evaluated on a test set consisting of $99$ objects.
• Figure 4: Numerical results on digits dataset with one hidden layer for CHANI with EWA, CHANI with PWA and CHANI with EWA and extra connections. The parameters $T=2000$, $M=40$, $N = 1357$ and $20$ realizations were made for each number of selected neurons. For CHANI with EWA: $\eta^1 = 3$, $\eta^2 = 0.002$. For CHANI with PWA: $b=2$ for each layer. For CHANI with EWA and extra connections: $\eta^1 = 3$, $\eta^2 = 0.007$, $\alpha=0.7$ and $\beta = 0.25$.

Theorems & Definitions (17)

• Definition 1: Ideal activities: layers and network
• Definition 2: Feasible weight family
• Theorem 3
• Theorem 9
• Corollary 10
• Corollary 12
• Theorem 14
• Corollary 15
• Theorem 18
• Proposition 19