Enhancing Disaster Resilience with UAV-Assisted Edge Computing: A Reinforcement Learning Approach to Managing Heterogeneous Edge Devices

TL;DR

The paper addresses sustaining edge sensing and computing in disaster scenarios where power and communications can fail. It introduces a UAV-assisted MEC framework combined with deep Q-network reinforcement learning to decide which heterogeneous edge device to visit and offload tasks, while enforcing energy and data-age constraints using a time-slotted model with variables such as $x_{ij}^t$, $E_j(t)$, and $D_j(t)$. The approach uses multiple reward formulations and trains a two-network DQN to maximize accumulated rewards, aiming to extend network lifetime and identify the most critical devices for maintenance. Case studies including rural and urban evacuations demonstrate that the DQN-based policy concentrates on devices near high-traffic pathways, improving data availability for responders. The work points to future directions such as deploying multiple UAVs, establishing ad-hoc mesh networks, and incorporating cost considerations for real-world disaster response.

Abstract

Edge sensing and computing is rapidly becoming part of intelligent infrastructure architecture leading to operational reliance on such systems in disaster or emergency situations. In such scenarios there is a high chance of power supply failure due to power grid issues, and communication system issues due to base stations losing power or being damaged by the elements, e.g., flooding, wildfires etc. Mobile edge computing in the form of unmanned aerial vehicles (UAVs) has been proposed to provide computation offloading from these devices to conserve their battery, while the use of UAVs as relay network nodes has also been investigated previously. This paper considers the use of UAVs with further constraints on power and connectivity to prolong the life of the network while also ensuring that the data is received from the edge nodes in a timely manner. Reinforcement learning is used to investigate numerous scenarios of various levels of power and communication failure. This approach is able to identify the device most likely to fail in a given scenario, thus providing priority guidance for maintenance personnel. The evacuations of a rural town and urban downtown area are also simulated to demonstrate the effectiveness of the approach at extending the life of the most critical edge devices.

Figures (4)

• Figure 1: Proposed System overview
• Figure 2: Example layout of edge devices in the region with the start position of the UAV at the origin. Distance is in meters, battery values are in Joules.
• Figure 3: Reinforcement learning training showing the increase in average episode length as the DQN learns the best policy.
• Figure 4: Road network with numbered edge devices. Traffic density on the roads is color coded in order of increasing density: blue, cyan, yellow, green, orange, red, magenta.