Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Traffic and Obstacle-aware UAV Positioning in Urban Environments Using Reinforcement Learning

Kamran Shafafi, Manuel Ricardo, Rui Campos

TL;DR

This work tackles UAV placement in obstacle-rich urban environments to deliver high-capacity wireless links by enforcing LoS to all ground users. It introduces RLTOPA, a Deep Q-Network–based algorithm that positions a single UAV by jointly considering ground-user locations, obstacle geometries, and traffic demands to maximize aggregate throughput while maintaining LoS. The method defines a discrete 3D action space, an observation space capturing UE/UAV coordinates and LoS counts, and a reward aligned with the proportion of UEs in LoS, and evaluates performance in ns-3 with an OpenAI Gym interface. Experimental results show substantial throughput and delay improvements (up to 95% throughput and 71% delay reductions) across multiple traffic scenarios, demonstrating RLTOPA’s potential for rapid, traffic-aware UAV deployments in urban crisis or congested settings. Future directions include dynamic scenarios, multi-UAV coordination, and integrating sensing/vision for improved UE and obstacle localization.

Abstract

Unmanned Aerial Vehicles (UAVs) are suited as cost-effective and adaptable platforms for carrying Wi-Fi Access Points (APs) and cellular Base Stations (BSs). Implementing aerial networks in disaster management scenarios and crowded areas can effectively enhance Quality of Service (QoS). In such environments, maintaining Line-of-Sight (LoS), especially at higher frequencies, is crucial for ensuring reliable communication networks with high capacity, particularly in environments with obstacles. The main contribution of this paper is a traffic- and obstacle-aware UAV positioning algorithm named Reinforcement Learning-based Traffic and Obstacle-aware Positioning Algorithm (RLTOPA), for such environments. RLTOPA determines the optimal position of the UAV by considering the positions of ground users, the coordinates of obstacles, and the traffic demands of users. This positioning aims to maximize QoS in terms of throughput by ensuring optimal LoS between ground users and the UAV. The network performance of the proposed solution, characterized in terms of mean delay and throughput, was evaluated using the ns- 3 simulator. The results show up to 95% improvement in aggregate throughput and 71% in delay without compromising fairness.

Traffic and Obstacle-aware UAV Positioning in Urban Environments Using Reinforcement Learning

TL;DR

This work tackles UAV placement in obstacle-rich urban environments to deliver high-capacity wireless links by enforcing LoS to all ground users. It introduces RLTOPA, a Deep Q-Network–based algorithm that positions a single UAV by jointly considering ground-user locations, obstacle geometries, and traffic demands to maximize aggregate throughput while maintaining LoS. The method defines a discrete 3D action space, an observation space capturing UE/UAV coordinates and LoS counts, and a reward aligned with the proportion of UEs in LoS, and evaluates performance in ns-3 with an OpenAI Gym interface. Experimental results show substantial throughput and delay improvements (up to 95% throughput and 71% delay reductions) across multiple traffic scenarios, demonstrating RLTOPA’s potential for rapid, traffic-aware UAV deployments in urban crisis or congested settings. Future directions include dynamic scenarios, multi-UAV coordination, and integrating sensing/vision for improved UE and obstacle localization.

Abstract

Unmanned Aerial Vehicles (UAVs) are suited as cost-effective and adaptable platforms for carrying Wi-Fi Access Points (APs) and cellular Base Stations (BSs). Implementing aerial networks in disaster management scenarios and crowded areas can effectively enhance Quality of Service (QoS). In such environments, maintaining Line-of-Sight (LoS), especially at higher frequencies, is crucial for ensuring reliable communication networks with high capacity, particularly in environments with obstacles. The main contribution of this paper is a traffic- and obstacle-aware UAV positioning algorithm named Reinforcement Learning-based Traffic and Obstacle-aware Positioning Algorithm (RLTOPA), for such environments. RLTOPA determines the optimal position of the UAV by considering the positions of ground users, the coordinates of obstacles, and the traffic demands of users. This positioning aims to maximize QoS in terms of throughput by ensuring optimal LoS between ground users and the UAV. The network performance of the proposed solution, characterized in terms of mean delay and throughput, was evaluated using the ns- 3 simulator. The results show up to 95% improvement in aggregate throughput and 71% in delay without compromising fairness.
Paper Structure (14 sections, 4 equations, 10 figures, 2 tables, 1 algorithm)

This paper contains 14 sections, 4 equations, 10 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. SYSTEM MODEL
  4. problem formulation
  5. RLTOPA Algorithm
  6. Action space
  7. Observation space
  8. Reward Function
  9. RLTOPA BLOCK DIAGRAM
  10. PERFORMANCE EVALUATION
  11. simulation setup
  12. simulation scenarios
  13. Simulation Results
  14. conclusions

Figures (10)

  • Figure 1: A simple scenario with a two-dimensional (2D) representation of the target positioning subspace ($S_p$) is illustrated. This subspace is determined by the intersection of spheres around each $UE_i$.
  • Figure 2: LoS and NLoS link representation based on ITU recommendation.
  • Figure 3: RLTOPA block diagram.
  • Figure 4: Scenario A -- four UE with homogeneous traffic demands: $B_0 = B_1 = B_2 = B_3$.
  • Figure 5: Scenario A -- four UE with heterogeneous traffic demands: $B_0 = B_1 = 2 \times B_2 = 2 \times B_3$.
  • ...and 5 more figures