Deep Reinforcement Learning for EH-Enabled Cognitive-IoT Under Jamming Attacks

TL;DR

<p>Addressing throughput optimization for energy-harvesting CIoT in underlay cognitive radio under jamming. The paper proposes a model-free MDP solved by a double deep Q-network (DDQN) augmented with interference-aware UCB (UCB-IA). Key contributions include problem formulation, DDQN-UCB-IA design, convergence insights, and extensive simulations showing superior performance over benchmarks. Demonstrates effective spectrum sharing and energy management under dynamic jamming, with practical implications for secure, self-sustaining CIoT networks.</p>

Abstract

In the evolving landscape of the Internet of Things (IoT), integrating cognitive radio (CR) has become a practical solution to address the challenge of spectrum scarcity, leading to the development of cognitive IoT (CIoT). However, the vulnerability of radio communications makes radio jamming attacks a key concern in CIoT networks. In this paper, we introduce a novel deep reinforcement learning (DRL) approach designed to optimize throughput and extend network lifetime of an energy-constrained CIoT system under jamming attacks. This DRL framework equips a CIoT device with the autonomy to manage energy harvesting (EH) and data transmission, while also regulating its transmit power to respect spectrum-sharing constraints. We formulate the optimization problem under various constraints, and we model the CIoT device's interactions within the channel as a model-free Markov decision process (MDP). The MDP serves as a foundation to develop a double deep Q-network (DDQN), designed to help the CIoT agent learn the optimal communication policy to navigate challenges such as dynamic channel occupancy, jamming attacks, and channel fading while achieving its goal. Additionally, we introduce a variant of the upper confidence bound (UCB) algorithm, named UCB-IA, which enhances the CIoT network's ability to efficiently navigate jamming attacks within the channel. The proposed DRL algorithm does not rely on prior knowledge and uses locally observable information such as channel occupancy, jamming activity, channel gain, and energy arrival to make decisions. Extensive simulations prove that our proposed DRL algorithm that utilizes the UCB-IA strategy surpasses existing benchmarks, allowing for a more adaptive, energy-efficient, and secure spectrum sharing in CIoT networks.

Figures (10)

• Figure 1: System Model
• Figure 2: The proposed DRL algorithm featuring the UCB-IA action exploration strategy.
• Figure 3: The CIoT Tx's ASR performance with $\epsilon$-greedy strategy across training episodes, comparison of different greediness value $\epsilon$.
• Figure 4: The CIoT Tx's ASR performance across training episodes, comparison of different strategies.
• Figure 5: The CIoT Tx's average achievable reward across training episodes under different strategies.