Neural Network-Based Bandit: A Medium Access Control for the IIoT Alarm Scenario

TL;DR

This work proposes a deep reinforcement learning-based distributed RA scheme, entitled Neural Network-Based Bandit (NNBB), for the IIoT alarm scenario, which uses a deep neural network to process this context and let devices determine their action.

Abstract

Efficient Random Access (RA) is critical for enabling reliable communication in Industrial Internet of Things (IIoT) networks. Herein, we propose a deep reinforcement learning based distributed RA scheme, entitled Neural Network-Based Bandit (NNBB), for the IIoT alarm scenario. In such a scenario, the devices may detect a common critical event, and the goal is to ensure the alarm information is delivered successfully from at least one device. The proposed NNBB scheme is implemented at each device, where it trains itself online and establishes implicit inter-device coordination to achieve the common goal. Devices can transmit simultaneously on multiple orthogonal channels and each possible transmission pattern constitutes a possible action for the NNBB, which uses a deep neural network to determine the action. Our simulation results show that as the number of devices in the network increases, so does the performance gain of the NNBB compared to the Multi-Armed Bandit (MAB) RA benchmark. For instance, NNBB experiences a 7% success rate drop when there are four channels and the number of devices increases from 10 to 60, while MAB faces a 25% drop.

Figures (10)

• Figure 1: IIoT alarm scenario. An alarm event at a certain location triggers some nearby devices, which then become active and must convey their sensed alarm information to the ExC. Here, the BS broadcasts the context, which is the aggregated pilot signal, to the active devices.
• Figure 2: Illustration of the proposed protocol exemplified for the case of $M\!=$ 2 orthogonal channels. First, each active device transmits a pilot signal to the BS. Then, the BS broadcasts the received pilot signal to devices (context broadcast). Finally, active devices transmit the alarm message after choosing their respective transmission patterns. In the structure of the alarm message, the alarm flag indicates the occurrence of an event, the header carries the metadata, and the payload contains relevant information for the ExC.
• Figure 3: DNN architecture. The input to the DNN is the received context, while its outputs are 2$^M$ parameterized action values, one for each possible action.
• Figure 4: Illustration of the MAC procedure followed by an active and inactive device after the occurrence of an alarm event.
• Figure 5: Variation in ${\textrm{MSE}_\textrm{sys}}$ during the training phase of NNBB for ${|\mathcal{N}|=20}$ and ${M=4}$.