DeepAir: A Multi-Agent Deep Reinforcement Learning Based Scheme for an Unknown User Location Problem

TL;DR

This study investigates the problem thoroughly and proposes a novel deep reinforcement learning (DRL) based scheme, DeepAir, which uses four main phases including sensing, localization, resource allocation, and multi-access edge computing (MEC) to provide the corresponding QoS requirements for the offloaded tasks without violating the maximum tolerable delay.

Abstract

The deployment of unmanned aerial vehicles (UAVs) in many different settings has provided various solutions and strategies for networking paradigms. Therefore, it reduces the complexity of the developments for the existing problems, which otherwise require more sophisticated approaches. One of those existing problems is the unknown user locations in an infrastructure-less environment in which users cannot connect to any communication device or computation-providing server, which is essential to task offloading in order to achieve the required quality of service (QoS). Therefore, in this study, we investigate this problem thoroughly and propose a novel deep reinforcement learning (DRL) based scheme, DeepAir. DeepAir considers all of the necessary steps including sensing, localization, resource allocation, and multi-access edge computing (MEC) to achieve QoS requirements for the offloaded tasks without violating the maximum tolerable delay. To this end, we use two types of UAVs including detector UAVs, and serving UAVs. We utilize detector UAVs as DRL agents which ensure sensing, localization, and resource allocation. On the other hand, we utilize serving UAVs to provide MEC features. Our experiments show that DeepAir provides a high task success rate by deploying fewer detector UAVs in the environment, which includes different numbers of users and user attraction points, compared to benchmark methods.

Figures (8)

• Figure 1: An air computing environment with different air platforms.
• Figure 2: Phases of DeepAir
• Figure 3: Depiction of the state of environment regarding the emitted signals
• Figure 4: Effect of learning rate on scores for each episode using 100 users.
• Figure 5: Effect of the number of users and serving UAVs on DeepAir.