Iteration over event space in time-to-first-spike spiking neural networks for Twitter bot classification

TL;DR

This study proposes a variant of a time-coding time-to-first-spike spiking neural network model with its neurons capable of generating spike trains in response to observed event sequences that is trained and evaluated on a Twitter bot detection task.

Abstract

This study proposes a framework that extends existing time-coding time-to-first-spike spiking neural network (SNN) models to allow processing information changing over time. We explain spike propagation through a model with multiple input and output spikes at each neuron, as well as design training rules for end-to-end backpropagation. This strategy enables us to process information changing over time. The model is trained and evaluated on a Twitter bot detection task where the time of events (tweets and retweets) is the primary carrier of information. This task was chosen to evaluate how the proposed SNN deals with spike train data composed of hundreds of events occurring at timescales differing by almost five orders of magnitude. The impact of various parameters on model properties, performance and training-time stability is analyzed.

Figures (7)

• Figure 1: Signal propagation rules in MIMO SNN. The flow of time is represented by an axis going from left to right, i.e., the earliest spike is at the left side of each subfigure. a) A presynaptic neuron that observes multiple input spikes can be represented as multiple virtual presynaptic neurons, each observing a single spike. All virtual presynaptic neurons have the same weight between them and the postsynaptic neuron, identical to the weight associated with the original connection before time-flattening. b) The ability of the postsynaptic neuron to produce spikes depends on all input spike trains from presynaptic neurons, as well as the previously generated output. This feedback loop imposed by the spike causality principle can be unrolled over time, where the postsynaptic neuron computation is repeated with the same input spike trains, but for different timestamps of the previously generated event. This cascade proceeds until it is impossible for the postsynaptic neuron to generate an output spike. The implicit output spike at $t_{\text{out}}^{[0]}=-\infty$ designates the initial state of the postsynaptic neuron, i.e., it has not generated a spike yet.
• Figure 2: Spike raster plot for models trained with two different values of $\tau_{\text{ref}}$ responding to the same input example.
• Figure 3: Simulated spike trains from a simple network with one input neuron and seven postsynaptic neurons. In blue: input spike train (the same in all rows), in orange: spikes generated by postsynaptic neurons. The shaded area denotes the refractory period after generating a spike. All output neurons have the same values of parameters $\tau_{\text{syn}}$, $\tau_{\text{ref}}$ and $V_{\text{thr}}$ with the only difference between them being the synaptic weight $w$. Note that if the weight it too large (in this case $w\ge3$), the neuron elicits a spike in response to every input event, unless it occurs during the refractory period, effectively repeating the input sequence. This means that a group of postsynaptic neurons is redundant because they produce almost identical spike trains. In such scenario, output sequence variability could be improved by adjusting the values of $\tau_{\text{syn}}$, $\tau_{\text{ref}}$ and $V_{\text{thr}}$ for each neuron individually.
• Figure 4: The result of binning the empirical density functions of the two classes (above - legitimate users; below - bots) into 10 bins over a given data split.
• Figure 5: A diagram of the preprocessing steps to obtain signals used to train a MIMO SNN on Twitter dataset.