Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UAV-assisted Internet of Vehicles: A Framework Empowered by Reinforcement Learning and Blockchain

Ahmed Alagha, Maha Kadadha, Rabeb Mizouni, Shakti Singh, Jamal Bentahar, Hadi Otrok

TL;DR

This work tackles the challenge of reliable, transparent relay selection and coordination in UAV-assisted IoV by introducing a Blockchain-enabled framework that couples relay assignment with decentralized multi-agent reinforcement learning (MDRL) for coordination. Relay selection is performed via a two-sided QoU (Quality-of-UAV) and QoV (Quality-of-Vehicle) mechanism implemented in a Quorum smart contract, with trained MDRL models distributed through IPFS to selected UAVs. Coordination among UAVs is formulated as a Markov Game and learned with Proximal Policy Optimization (PPO) under Centralized Learning, Decentralized Execution (CLDE), using a CNN-based actor and a team-oriented reward to maximize connectivity and coverage. The framework is evaluated through simulations showing improved QoU/QoV and superior scalable coordination compared with centralized baselines, and a cost analysis demonstrates practical deployment feasibility. Overall, the proposed approach offers a transparent, scalable pathway to autonomous UAV-IoV networks capable of adapting to dynamic vehicle distributions and operational requirements.

Abstract

This paper addresses the challenges of selecting relay nodes and coordinating among them in UAV-assisted Internet-of-Vehicles (IoV). The selection of UAV relay nodes in IoV employs mechanisms executed either at centralized servers or decentralized nodes, which have two main limitations: 1) the traceability of the selection mechanism execution and 2) the coordination among the selected UAVs, which is currently offered in a centralized manner and is not coupled with the relay selection. Existing UAV coordination methods often rely on optimization methods, which are not adaptable to different environment complexities, or on centralized deep reinforcement learning, which lacks scalability in multi-UAV settings. Overall, there is a need for a comprehensive framework where relay selection and coordination are coupled and executed in a transparent and trusted manner. This work proposes a framework empowered by reinforcement learning and Blockchain for UAV-assisted IoV networks. It consists of three main components: a two-sided UAV relay selection mechanism for UAV-assisted IoV, a decentralized Multi-Agent Deep Reinforcement Learning (MDRL) model for autonomous UAV coordination, and a Blockchain implementation for transparency and traceability in the interactions between vehicles and UAVs. The relay selection considers the two-sided preferences of vehicles and UAVs based on the Quality-of-UAV (QoU) and the Quality-of-Vehicle (QoV). Upon selection of relay UAVs, the decentralized coordination between them is enabled through an MDRL model trained to control their mobility and maintain the network coverage and connectivity using Proximal Policy Optimization (PPO). The evaluation results demonstrate that the proposed selection and coordination mechanisms improve the stability of the selected relays and maximize the coverage and connectivity achieved by the UAVs.

UAV-assisted Internet of Vehicles: A Framework Empowered by Reinforcement Learning and Blockchain

TL;DR

This work tackles the challenge of reliable, transparent relay selection and coordination in UAV-assisted IoV by introducing a Blockchain-enabled framework that couples relay assignment with decentralized multi-agent reinforcement learning (MDRL) for coordination. Relay selection is performed via a two-sided QoU (Quality-of-UAV) and QoV (Quality-of-Vehicle) mechanism implemented in a Quorum smart contract, with trained MDRL models distributed through IPFS to selected UAVs. Coordination among UAVs is formulated as a Markov Game and learned with Proximal Policy Optimization (PPO) under Centralized Learning, Decentralized Execution (CLDE), using a CNN-based actor and a team-oriented reward to maximize connectivity and coverage. The framework is evaluated through simulations showing improved QoU/QoV and superior scalable coordination compared with centralized baselines, and a cost analysis demonstrates practical deployment feasibility. Overall, the proposed approach offers a transparent, scalable pathway to autonomous UAV-IoV networks capable of adapting to dynamic vehicle distributions and operational requirements.

Abstract

This paper addresses the challenges of selecting relay nodes and coordinating among them in UAV-assisted Internet-of-Vehicles (IoV). The selection of UAV relay nodes in IoV employs mechanisms executed either at centralized servers or decentralized nodes, which have two main limitations: 1) the traceability of the selection mechanism execution and 2) the coordination among the selected UAVs, which is currently offered in a centralized manner and is not coupled with the relay selection. Existing UAV coordination methods often rely on optimization methods, which are not adaptable to different environment complexities, or on centralized deep reinforcement learning, which lacks scalability in multi-UAV settings. Overall, there is a need for a comprehensive framework where relay selection and coordination are coupled and executed in a transparent and trusted manner. This work proposes a framework empowered by reinforcement learning and Blockchain for UAV-assisted IoV networks. It consists of three main components: a two-sided UAV relay selection mechanism for UAV-assisted IoV, a decentralized Multi-Agent Deep Reinforcement Learning (MDRL) model for autonomous UAV coordination, and a Blockchain implementation for transparency and traceability in the interactions between vehicles and UAVs. The relay selection considers the two-sided preferences of vehicles and UAVs based on the Quality-of-UAV (QoU) and the Quality-of-Vehicle (QoV). Upon selection of relay UAVs, the decentralized coordination between them is enabled through an MDRL model trained to control their mobility and maintain the network coverage and connectivity using Proximal Policy Optimization (PPO). The evaluation results demonstrate that the proposed selection and coordination mechanisms improve the stability of the selected relays and maximize the coverage and connectivity achieved by the UAVs.

Paper Structure

This paper contains 20 sections, 4 equations, 9 figures, 7 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. IoV Relay Selection
  4. Blockchain for Trusted Execution
  5. UAV Coordination
  6. Proposed System
  7. Proposed Blockchain-based Framework
  8. UAV-assisted IoV Construction
  9. MDRL Approach for UAV Coordination
  10. MDRL Formulation and Policy Optimization
  11. Action Space
  12. Policy Architecture
  13. Reward Function
  14. Process Sequence Diagram
  15. Simulation and Evaluation
  16. ...and 5 more sections

Figures (9)

  • Figure 1: The different layers of the system in the proposed framework
  • Figure 2: The original observations acquired by a UAV, in a team of 3 UAVs, and their reduced counterparts (local in green and global in red).
  • Figure 3: The architecture used for the actor and critic networks. While both networks have the same architecture configuration, they are two separate networks with two different outputs.
  • Figure 4: The overall process diagram
  • Figure 5: Number of vehicles accepted per UAV and the percentage of selected UAVs.
  • ...and 4 more figures