Continual Deep Reinforcement Learning to Prevent Catastrophic Forgetting in Jamming Mitigation

TL;DR

This work tackles catastrophic forgetting in DRL-based anti-jamming under non-stationary jammer patterns. It adopts PackNet-based continual learning to preserve past jammer-pattern knowledge while learning new ones, using parameter isolation, pruning, and selective retraining with $DQN$ and $SAC$ evaluated on Env1/Env2/Env3, incorporating a spectral-efficiency reward. The key contributions include a systematic continual learning framework, empirical demonstrations of reduced forgetting and improved retention, and faster, more reliable convergence for sequential jamming scenarios. The approach enhances the adaptability and robustness of wireless networks facing dynamic, adversarial jamming, enabling sustained performance in realistic deployments.

Abstract

Deep Reinforcement Learning (DRL) has been highly effective in learning from and adapting to RF environments and thus detecting and mitigating jamming effects to facilitate reliable wireless communications. However, traditional DRL methods are susceptible to catastrophic forgetting (namely forgetting old tasks when learning new ones), especially in dynamic wireless environments where jammer patterns change over time. This paper considers an anti-jamming system and addresses the challenge of catastrophic forgetting in DRL applied to jammer detection and mitigation. First, we demonstrate the impact of catastrophic forgetting in DRL when applied to jammer detection and mitigation tasks, where the network forgets previously learned jammer patterns while adapting to new ones. This catastrophic interference undermines the effectiveness of the system, particularly in scenarios where the environment is non-stationary. We present a method that enables the network to retain knowledge of old jammer patterns while learning to handle new ones. Our approach substantially reduces catastrophic forgetting, allowing the anti-jamming system to learn new tasks without compromising its ability to perform previously learned tasks effectively. Furthermore, we introduce a systematic methodology for sequentially learning tasks in the anti-jamming framework. By leveraging continual DRL techniques based on PackNet, we achieve superior anti-jamming performance compared to standard DRL methods. Our proposed approach not only addresses catastrophic forgetting but also enhances the adaptability and robustness of the system in dynamic jamming environments. We demonstrate the efficacy of our method in preserving knowledge of past jammer patterns, learning new tasks efficiently, and achieving superior anti-jamming performance compared to traditional DRL approaches.

Figures (4)

• Figure 1: DQN performance in Scenario 1.
• Figure 2: SAC performance in Scenario 1.
• Figure 3: Mean reward performance of three agents in Scenario 2.
• Figure 4: Mean reward of the three DRL agents in Scenario 3B.