Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks

TL;DR

This work introduces the Plasticity-Driven Learning Framework (PDLF) for Spiking Neural Networks, shifting learning from static weight optimization to discovering flexible plasticity rules. By combining Synaptic Cooperation Plasticity (SCP) and Presynaptic-Dependent Plasticity (PDP) and optimizing their parameters with an Evolutionary Strategy, PDLF enables SNNs to adapt dynamically to tasks. The approach yields improved working memory, multi-task learning, and generalization, with memories encoded directly into synaptic weights and robust performance under simulated neural injuries. These results suggest a path toward more adaptable, energy-efficient AI that more closely mirrors the brain's continual learning capabilities.

Abstract

The evolution of the human brain has led to the development of complex synaptic plasticity, enabling dynamic adaptation to a constantly evolving world. This progress inspires our exploration into a new paradigm for Spiking Neural Networks (SNNs): a Plasticity-Driven Learning Framework (PDLF). This paradigm diverges from traditional neural network models that primarily focus on direct training of synaptic weights, leading to static connections that limit adaptability in dynamic environments. Instead, our approach delves into the heart of synaptic behavior, prioritizing the learning of plasticity rules themselves. This shift in focus from weight adjustment to mastering the intricacies of synaptic change offers a more flexible and dynamic pathway for neural networks to evolve and adapt. Our PDLF does not merely adapt existing concepts of functional and Presynaptic-Dependent Plasticity but redefines them, aligning closely with the dynamic and adaptive nature of biological learning. This reorientation enhances key cognitive abilities in artificial intelligence systems, such as working memory and multitasking capabilities, and demonstrates superior adaptability in complex, real-world scenarios. Moreover, our framework sheds light on the intricate relationships between various forms of plasticity and cognitive functions, thereby contributing to a deeper understanding of the brain's learning mechanisms. Integrating this groundbreaking plasticity-centric approach in SNNs marks a significant advancement in the fusion of neuroscience and artificial intelligence. It paves the way for developing AI systems that not only learn but also adapt in an ever-changing world, much like the human brain.

Figures (6)

• Figure 1: Diagram of PDLF. Top: By combining Synaptic Cooperation Plasticity (SCP) and Presynaptic-Dependent Plasticity (PDP), neurons can achieve diverse and heterogeneous plasticity. Bottom: Agents with PDLF learn plasticity rather than directly adjusting weights. Different forms of synaptic plasticity can be formed between neurons, enabling better multi-task learning. Plasticity helps the agents dynamically adjust weights and learn previously unseen scenarios during training, even without explicit reward signals.
• Figure 2: Design of the WM experiment and the impact of PDLF on WM. A. Schematic of the copying task. The SNNs first receive a sequence of motion stimuli, with each stimulus lasting $200$ ms, followed by a delay period of varying lengths, and finally, a test stimulus of the same duration as the sample stimulus. The SNNs are required to reproduce the stimuli from the first phase to receive the test stimulus. B. Performance comparison between SNNs with plasticity and trained with direct weights. SNNs with plasticity show faster convergence, longer memory duration, and greater memory capacity. 'Len' refers to the length of stimulus samples, while 'Lat' refers to the number of steps in the delay period. C. Synaptic weights after different motion stimulus inputs when the number of motion samples is $8$. SNNs with plasticity can form distinct connection weights for different stimuli. The left side of the dashed line shows the input weights associated with the stimulus, and the right side shows the output weights. D. Neuron states at different stages in SNNs trained directly with weights and those with PDLF. Directly trained SNNs require neural activity during the delay period to maintain memory. Resetting the membrane potential to $0$ after the input stimulus leads to a chance-level memory accuracy, resulting in memory loss. In contrast, SNNs with plasticity can encode input stimuli into synaptic weights, demonstrating stronger memory functionality. E. Visualization of the firing rates at different stages and the average spike traces for SNNs using different strategies. SNNs with plasticity can maintain lower firing rates.
• Figure 3: PDLF's performance in multi-task reinforcement learning tasks and the influence of different plasticity attributes on performance.A. Illustration of multi-task reinforcement learning. The agent is required to utilize a singular network to simultaneously learn multiple tasks with distinct objectives or even entirely opposing ones. The objectives of the tasks are treated as observations for the agent. They are inputted to the SNNs along with other observations such as joint positions, velocities, etc. B. Training curves of agents with PDLF versus those trained directly on weights. In these multi-task RL tasks, agents need to learn to move towards different directions (ant_dir, swimmer_dir), at varying speeds (halfcheetah_vel, hopper_vel), and to different locations (fetch, ur5e). Agents with PDLF maintain dynamic synaptic weights, learn characteristics of different tasks, and hence achieve superior performance in multi-task challenges. C. Ablation analysis of different plasticity mechanisms. Different colors represent training curves with some form of plasticity (SCP or PDP) removed. After removing PDP, SNNs diverge due to the loss of the equilibrium mechanism. Both SCP and PDP play significant roles in enhancing agent performance. D. The change curve of a synapse's PDLF during the training process. Through evolutionary strategies, agents learn to adjust their plasticity. E. During the training process, at different inter-spike intervals of pre-synaptic and post-synaptic neurons, the impact of the plasticity of agents from different generations on weights. F. The specific functions of plasticity in agents from different generations as shown in E.
• Figure 4: Performance under temporary and permanent nerve injury.A. The agent's performance in the face of temporary neural damage. At the $500$th step, all synaptic weights were reset to $0$ to simulate a sudden neural system injury, and this condition lasted for $50$ steps. Agents with plasticity were able to recover from this temporary loss, demonstrating better robustness. B. Network weights at different times before and after temporary damage. The synaptic weights of the input layer are shown above the dashed line, while the readout layer weights are below the dashed line. Even if the agent loses all synaptic weights due to temporary damage, it can still recover these weights based on its plasticity and input stimuli. C. The agent's performance in the face of permanent neuronal damage of varying degrees. At the start of the test, neurons were blocked at different proportions, their synaptic weights set to $0$, and could not be updated, simulating permanent neural network damage. Agents with plasticity performed better and exhibited stronger robustness when dealing with such permanent damage.
• Figure 5: A. Performance of different agents in trained tasks and tasks not seen during training. Agents with plasticity can generalize well to unseen tasks, while agents trained directly on weights have difficulty generalizing to unseen test tasks due to their weights being fixed during testing. B. Low-dimensional embeddings of neuronal states during reinforcement learning tasks, differentiated by training strategy. Each point corresponds to the state of the hidden layer neurons at a specific time step. The color coding signifies distinct tasks. Agents that possess plasticity demonstrate an enhanced capability to distinguish between different tasks. Moreover, the neuronal states associated with identical tasks exhibit the intriguing property of forming a manifold within the high-dimensional space.