Adapting to time: Why nature may have evolved a diverse set of neurons

TL;DR

It is found that adapting conduction delays is crucial for solving all test conditions under tight resource constraints, and rich and adaptable dynamics may be the key for solving temporally structured tasks efficiently in evolving organisms.

Abstract

Brains have evolved diverse neurons with varying morphologies and dynamics that impact temporal information processing. In contrast, most neural network models use homogeneous units that vary only in spatial parameters (weights and biases). To explore the importance of temporal parameters, we trained spiking neural networks on tasks with varying temporal complexity, holding different parameter subsets constant. We found that adapting conduction delays is crucial for solving all test conditions under tight resource constraints. Remarkably, these tasks can be solved using only temporal parameters (delays and time constants) with constant weights. In more complex spatio-temporal tasks, an adaptable bursting parameter was essential. Overall, allowing adaptation of both temporal and spatial parameters enhances network robustness to noise, a vital feature for biological brains and neuromorphic computing systems. Our findings suggest that rich and adaptable dynamics may be the key for solving temporally structured tasks efficiently in evolving organisms, which would help explain the diverse physiological properties of biological neurons.

Figures (6)

• Figure 1: Schematic of the framework for this study. (A) The network architecture and the input-output encoding pair. (B) The neuron model illustrating the adaptable parameters and computation of the somatic voltage. (C) Steps of the evolutionary algorithm, which includes: initialization, evaluation, sorting by loss, selection of elites and the generation of a new child population.
• Figure 2: Effect of the input-output encoding and the co-mutated parameters on the search speed and availability of solutions for various logic problems. This effect is conveyed through the number of generations needed to find a solution, where the colour 'tan' means no solutions found. This is performed for (A) [-2, 2] mV and (B) [-1, 1] mV weight clipping range during evolution. (C) $W\hbox{$\tau$}_c$ only solutions can simulate delays. Left, the spiking plot for a sample XNOR problem and right, an enlarged view for the voltage traces with their sum. Abbreviations code, $W$: weights, $\hbox{$\tau$}_c$: time constants, $D$: delays. For more details, see Table A in S1 Text and the related text.
• Figure 3: Properties of semi-temporal logic problems. (A) Solutions characteristics; demonstrating the impact of the co-mutated parameters and the input-output encoding on the ratios of excitatory vs. inhibitory connections and long vs. short time constants. (B) The weight clipping range dictates the modality of computation and excitatory/inhibitory ratio. Left, distribution of weights in the network at various weight clipping ranges. Right, spiking raster plots illustrating the network behavior when the weight clipping range is below (top row) and above threshold (bottom row). (C) Delays and time constants alone can solve all logic problems with constant weights (1 mV). The heatmaps show the values of change in delays and time constants for the particular XOR problem in the accompanying spiking raster plots (right), while the histograms show the distributions for both parameters aggregated across all logic problems. Regarding delays, we mutate/adapt the change in delays and add it to a default value to acquire the total axonal delays (D) Weights and time constant distributions as a function of the output code. For more details, see Tables B, C and D in S1 Text and the related text.
• Figure 4: The relationship between time constants, input encoding and weight clipping range in weights-delays mutated solutions for the XOR problem. (A) This interplay is captured through the average number of generations needed to reach a zero loss (perfect) solution, where each grid cell is the average of five solutions. (B) Average change in delays as a function of the weight clipping range and time constants. Each bar is the average of several populations of solutions that solve the XOR, XNOR, OR, and AND logic problems. For more details, see Tables E and F in S1 Text and the related text.
• Figure 5: The effect of noise and uncertainty in inputs and weights. (A) Impact of additive noise and spike jitter in the input, quantified by the minimum loss achieved within 100 generations. This is shown for weights-delays, weights-time constants, delays-time constants and weights-delays-time constants mutated solutions. (B) Robustness of solutions trained to minimize noise in weights quantified by the minimum loss in 1000 generations, with each loss averaged over 100 trials. This is shown for weights only, weights-delays, weights-time constants and weights-delays-time constants mutated solutions. For more details, see Tables G and H in S1 Text and the related text.