MTSpark: Enabling Multi-Task Learning with Spiking Neural Networks for Generalist Agents

TL;DR

This paper addresses catastrophic forgetting in multi-task reinforcement learning by introducing MTSpark, a Spiking Neural Network (SNN) framework that uses active dendrites and task-context signals to form task-conditioned sub-networks. Building on a Deep Spiking Q-Network (DSQN) backbone, MTSpark incorporates a dueling architecture (MTSpark_ADD) and two variants (MTSpark_AD) to balance performance and resource use, achieving high scores on Atari games and strong image-classification results. The approach emphasizes energy efficiency and hardware suitability while maintaining a unified parameter set across tasks, enabling efficient multi-task adaptation without extensive architectural growth. The results demonstrate competitive, often human-level, performance across multiple tasks and datasets, underscoring the potential of combining RL with SNNs for generalist agents and energy-efficient neuromorphic deployment.

Abstract

Currently, state-of-the-art RL methods excel in single-task settings, but they still struggle to generalize across multiple tasks due to catastrophic forgetting challenges, where previously learned tasks are forgotten as new tasks are introduced. This multi-task learning capability is significantly important for generalist agents, where adaptation features are highly required (e.g., autonomous robots). On the other hand, Spiking Neural Networks (SNNs) have emerged as alternative energy-efficient neural network algorithms due to their sparse spike-based operations. Toward this, we propose MTSpark, a novel methodology to enable multi-task RL using spiking networks. Specifically, MTSpark develops a Deep Spiking Q-Network (DSQN) with active dendrites and dueling structure by leveraging task-specific context signals. Specifically, each neuron computes task-dependent activations that dynamically modulate inputs, forming specialized sub-networks for each task. Moreover, this bioplausible network model also benefits from SNNs, enhancing energy efficiency and making the model suitable for hardware implementation. Experimental results show that, our MTSpark effectively learns multiple tasks with higher performance compared to the state-of-the-art. Specifically, MTSpark successfully achieves high score in three Atari games (i.e., Pong: -5.4, Breakout: 0.6, and Enduro: 371.2), reaching human-level performance (i.e., Pong: -3, Breakout: 31, and Enduro: 368), where state-of-the-art struggle to achieve. In addition, our MTSpark also shows better accuracy in image classification tasks than the state-of-the-art. These results highlight the potential of our MTSpark methodology to develop generalist agents that can learn multiple tasks by leveraging both RL and SNN concepts.

Figures (10)

• Figure 1: Multi-task learning performance of the state-of-the-art RL-based methods for DNN (i.e., DQN Playing_Atari) and SNN (i.e., DSQN DSQN), and our MTSpark methodology, across 3 Atari games (Pong, Breakout, and Enduro). These results show that MTSpark achieves high performance in all tasks, while the state-of-the-art only perform well in certain tasks.
• Figure 2: The overview of our MTSpark methodology, showing three key steps: spiking neuron development with active dendrites, network architecture design, and effective training strategy.
• Figure 3: The overview of a DQN architecture.
• Figure 4: The proposed integrate-and-fire (IF) neuron model is enhanced using active dendrites with context signals.
• Figure 5: The overview of a DSQN architecture.