Population-Coded Spiking Neural Networks for High-Dimensional Robotic Control

TL;DR

This work addresses energy-efficient, high-dimensional robotic control by integrating population-coded Spiking Neural Networks (SNNs) with Deep Reinforcement Learning (DRL). The PopSAN policy encodes observations into neural populations and processes them via a multi-layer SNN, trained with extended spatiotemporal backpropagation to optimize DRL objectives. On Isaac Gym PixMC tasks with a Franka arm, the approach achieves substantial energy savings (up to 96.10% vs ANN) while maintaining competitive control performance and robust finger/height tracking. The results demonstrate PopSAN as a viable, scalable solution for real-world, resource-constrained robotics, enabling faster, energy-efficient operation suitable for neuromorphic deployment and future multimodal integrations.

Abstract

Energy-efficient and high-performance motor control remains a critical challenge in robotics, particularly for high-dimensional continuous control tasks with limited onboard resources. While Deep Reinforcement Learning (DRL) has achieved remarkable results, its computational demands and energy consumption limit deployment in resource-constrained environments. This paper introduces a novel framework combining population-coded Spiking Neural Networks (SNNs) with DRL to address these challenges. Our approach leverages the event-driven, asynchronous computation of SNNs alongside the robust policy optimization capabilities of DRL, achieving a balance between energy efficiency and control performance. Central to this framework is the Population-coded Spiking Actor Network (PopSAN), which encodes high-dimensional observations into neuronal population activities and enables optimal policy learning through gradient-based updates. We evaluate our method on the Isaac Gym platform using the PixMC benchmark with complex robotic manipulation tasks. Experimental results on the Franka robotic arm demonstrate that our approach achieves energy savings of up to 96.10% compared to traditional Artificial Neural Networks (ANNs) while maintaining comparable control performance. The trained SNN policies exhibit robust finger position tracking with minimal deviation from commanded trajectories and stable target height maintenance during pick-and-place operations. These results position population-coded SNNs as a promising solution for energy-efficient, high-performance robotic control in resource-constrained applications, paving the way for scalable deployment in real-world robotics systems.

Figures (6)

• Figure 1: Fully spike-based control on robotic arms (Franka) in Isaac Gym
• Figure 2: The observations are initially encoded by the encoder as $n$ independent distributions that are uniformly distributed over the observation range. After encoding, the population processes the distributions, resulting in spike generation. The neurons in the input populations encode each observation dimension and drive a multi-layered, fully connected SNN. During forward timesteps in PopSAN, the activities of each output population are decoded to determine the corresponding action dimension. This implies that the neural network receives observations, processes them using the SNN, and decodes the resulting activities to determine the appropriate action for the specific situation.
• Figure 3: Comparison of Mean Success Rate for SNN and ANN during training.
• Figure 4: Comparison of Mean Episode Length for SNN and ANN during training.
• Figure 5: Comparison of Mean Reward for SNN and ANN during training.