Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Search and Rescue UAV Connectivity in Challenging Terrain through Multi Q-Learning

Mohammed M. H. Qazzaz, Syed A. R. Zaidi, Desmond C. McLernon, Abdelaziz Salama, Aubida A. Al-Hameed

TL;DR

The paper tackles the challenge of maintaining robust cellular connectivity for SAR UAVs operating in rugged terrain by deploying a multi Q-learning framework with two specialized agents: a Strategic Planning Agent for efficient, collision-aware path planning and a Real-time Adaptive Agent for connectivity optimization across multiple frequency bands. A dual-model decision mechanism allows the UAV to balance trajectory efficiency with link reliability, using a COST Hata propagation model to simulate wireless performance. Evaluations across varying obstacle densities and frequency bands demonstrate high arrival success in favorable environments, with observable trade-offs between path efficiency and connectivity, particularly at higher frequencies where path loss increases. Overall, the study demonstrates that multi-agent reinforcement learning can enhance UAV autonomy and communication reliability in challenging SAR contexts, pointing to future work in dynamic obstacles, weather integration, and coordinated multi-UAV strategies.

Abstract

Using Unmanned Aerial Vehicles (UAVs) in Search and rescue operations (SAR) to navigate challenging terrain while maintaining reliable communication with the cellular network is a promising approach. This paper suggests a novel technique employing a reinforcement learning multi Q-learning algorithm to optimize UAV connectivity in such scenarios. We introduce a Strategic Planning Agent for efficient path planning and collision awareness and a Real-time Adaptive Agent to maintain optimal connection with the cellular base station. The agents trained in a simulated environment using multi Q-learning, encouraging them to learn from experience and adjust their decision-making to diverse terrain complexities and communication scenarios. Evaluation results reveal the significance of the approach, highlighting successful navigation in environments with varying obstacle densities and the ability to perform optimal connectivity using different frequency bands. This work paves the way for enhanced UAV autonomy and enhanced communication reliability in search and rescue operations.

Optimizing Search and Rescue UAV Connectivity in Challenging Terrain through Multi Q-Learning

TL;DR

The paper tackles the challenge of maintaining robust cellular connectivity for SAR UAVs operating in rugged terrain by deploying a multi Q-learning framework with two specialized agents: a Strategic Planning Agent for efficient, collision-aware path planning and a Real-time Adaptive Agent for connectivity optimization across multiple frequency bands. A dual-model decision mechanism allows the UAV to balance trajectory efficiency with link reliability, using a COST Hata propagation model to simulate wireless performance. Evaluations across varying obstacle densities and frequency bands demonstrate high arrival success in favorable environments, with observable trade-offs between path efficiency and connectivity, particularly at higher frequencies where path loss increases. Overall, the study demonstrates that multi-agent reinforcement learning can enhance UAV autonomy and communication reliability in challenging SAR contexts, pointing to future work in dynamic obstacles, weather integration, and coordinated multi-UAV strategies.

Abstract

Using Unmanned Aerial Vehicles (UAVs) in Search and rescue operations (SAR) to navigate challenging terrain while maintaining reliable communication with the cellular network is a promising approach. This paper suggests a novel technique employing a reinforcement learning multi Q-learning algorithm to optimize UAV connectivity in such scenarios. We introduce a Strategic Planning Agent for efficient path planning and collision awareness and a Real-time Adaptive Agent to maintain optimal connection with the cellular base station. The agents trained in a simulated environment using multi Q-learning, encouraging them to learn from experience and adjust their decision-making to diverse terrain complexities and communication scenarios. Evaluation results reveal the significance of the approach, highlighting successful navigation in environments with varying obstacle densities and the ability to perform optimal connectivity using different frequency bands. This work paves the way for enhanced UAV autonomy and enhanced communication reliability in search and rescue operations.
Paper Structure (14 sections, 3 equations, 5 figures, 2 tables, 3 algorithms)

This paper contains 14 sections, 3 equations, 5 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Motivation
  3. Related Works
  4. Contributions and Problem Statement
  5. Organisation
  6. Reinforcement Learning (RL) Algorithm
  7. System Model
  8. Model Environement
  9. Model Impelementation
  10. Strategic Planning Agent
  11. Real-time Adaptive Agent
  12. Dual Model Decision-Making Approach
  13. Results and Discussion
  14. Conclusions

Figures (5)

  • Figure 1: Reinforcement Learning Model munoz2019deep
  • Figure 2: Supposed Model 3D Environement
  • Figure 3: Environement Obstacles Distribution Scenarios
  • Figure 4: Strategic Planning Agent Training Rewards
  • Figure 5: Real-time Adaptive Agent Training Rewards