Error Amplification Limits ANN-to-SNN Conversion in Continuous Control

TL;DR

This work proposes Cross-Step Residual Potential Initialization (CRPI), a lightweight training-free mechanism that carries over residual membrane potentials across decision steps to suppress temporally correlated errors in Spiking Neural Networks.

Abstract

Spiking Neural Networks (SNNs) can achieve competitive performance by converting already existing well-trained Artificial Neural Networks (ANNs), avoiding further costly training. This property is particularly attractive in Reinforcement Learning (RL), where training through environment interaction is expensive and potentially unsafe. However, existing conversion methods perform poorly in continuous control, where suitable baselines are largely absent. We identify error amplification as the key cause: small action approximation errors become temporally correlated across decision steps, inducing cumulative state distribution shift and severe performance degradation. To address this issue, we propose Cross-Step Residual Potential Initialization (CRPI), a lightweight training-free mechanism that carries over residual membrane potentials across decision steps to suppress temporally correlated errors. Experiments on continuous control benchmarks with both vector and visual observations demonstrate that CRPI can be integrated into existing conversion pipelines and substantially recovers lost performance. Our results highlight continuous control as a critical and challenging benchmark for ANN-to-SNN conversion, where small errors can be strongly amplified and impact performance.

Figures (7)

• Figure 1: Challenges of ANN-to-SNN conversion across different task categories. (a) Classification accuracy on ImageNet huang2025differential. (b) Average returns in discrete control tasks on Atari DQN2ANN1. (c) Relative returns in continuous control tasks, averaged over six environments from the DeepMind Control Suite. Additional results in the experimental section confirm that the performance degradation is consistent across tasks. (d) Illustration of error accumulation and amplification, where trajectories generated by converted SNNs progressively diverge from those of the original ANN policies.
• Figure 2: Analysis of performance degradation in ANN-to-SNN conversion in the HalfCheetah-v4 environment. The ANN policy is trained with TD3 for $3$ million environment steps and converted using IF neurons with $8$ simulation steps. (a) Expected returns under different combinations of policies and state distributions. (b) t-SNE visualization of state trajectories induced by ANN and converted SNN policies, revealing significant distribution divergence.
• Figure 3: (a) One-dimensional visualization of state evolution over decision steps for ANN and converted SNN policies in the HalfCheetah-v4 environment, obtained by projecting paired trajectories onto the first principal component via PCA. (b) Average reward per decision step for ANN and converted SNN policies in Hopper-v4 (left) and Walker2d-v4 (right). Shaded regions denote half a standard deviation. All curves are uniformly smoothed for clarity. The ANN was trained with TD3 for $3$ million environment steps, and the SNN uses IF neurons.
• Figure 4: Cosine similarity of residual membrane potential and action errors across consecutive decision steps under different values of $\alpha$. Results are obtained on MuJoCo environments using TD3 and IF neurons with $16$ simulation steps.
• Figure 5: Relative performance on DeepMind Control tasks under different correlation parameter $\alpha$, using IF neurons with $32$ simulation steps. Performance is normalized by the corresponding ANN return. Curves are uniformly smoothed for visualization.