Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vision-Based Deep Reinforcement Learning of UAV Autonomous Navigation Using Privileged Information

Junqiao Wang, Zhongliang Yu, Dong Zhou, Jiaqi Shi, Runran Deng

TL;DR

This work addresses autonomous UAV navigation under partial observability by integrating deep reinforcement learning with privileged information. The proposed DPRL framework employs an asymmetric Actor-Critic architecture and asynchronous multi-agent exploration to utilize privileged perception data (noiseless depth, accurate localization, and an obstacle map) during training, accelerating convergence and improving robustness. In TD3-based training, DPRL achieves faster learning and higher performance than TD3 and EGO-Planner-v2, with ablations confirming the value of privileged information and a 4D action space for efficient obstacle avoidance. The results demonstrate significant potential for real-world deployment and simulation-to-real transfer, particularly in complex, unknown environments where perception is imperfect.

Abstract

The capability of UAVs for efficient autonomous navigation and obstacle avoidance in complex and unknown environments is critical for applications in agricultural irrigation, disaster relief and logistics. In this paper, we propose the DPRL (Distributed Privileged Reinforcement Learning) navigation algorithm, an end-to-end policy designed to address the challenge of high-speed autonomous UAV navigation under partially observable environmental conditions. Our approach combines deep reinforcement learning with privileged learning to overcome the impact of observation data corruption caused by partial observability. We leverage an asymmetric Actor-Critic architecture to provide the agent with privileged information during training, which enhances the model's perceptual capabilities. Additionally, we present a multi-agent exploration strategy across diverse environments to accelerate experience collection, which in turn expedites model convergence. We conducted extensive simulations across various scenarios, benchmarking our DPRL algorithm against the state-of-the-art navigation algorithms. The results consistently demonstrate the superior performance of our algorithm in terms of flight efficiency, robustness and overall success rate.

Vision-Based Deep Reinforcement Learning of UAV Autonomous Navigation Using Privileged Information

TL;DR

This work addresses autonomous UAV navigation under partial observability by integrating deep reinforcement learning with privileged information. The proposed DPRL framework employs an asymmetric Actor-Critic architecture and asynchronous multi-agent exploration to utilize privileged perception data (noiseless depth, accurate localization, and an obstacle map) during training, accelerating convergence and improving robustness. In TD3-based training, DPRL achieves faster learning and higher performance than TD3 and EGO-Planner-v2, with ablations confirming the value of privileged information and a 4D action space for efficient obstacle avoidance. The results demonstrate significant potential for real-world deployment and simulation-to-real transfer, particularly in complex, unknown environments where perception is imperfect.

Abstract

The capability of UAVs for efficient autonomous navigation and obstacle avoidance in complex and unknown environments is critical for applications in agricultural irrigation, disaster relief and logistics. In this paper, we propose the DPRL (Distributed Privileged Reinforcement Learning) navigation algorithm, an end-to-end policy designed to address the challenge of high-speed autonomous UAV navigation under partially observable environmental conditions. Our approach combines deep reinforcement learning with privileged learning to overcome the impact of observation data corruption caused by partial observability. We leverage an asymmetric Actor-Critic architecture to provide the agent with privileged information during training, which enhances the model's perceptual capabilities. Additionally, we present a multi-agent exploration strategy across diverse environments to accelerate experience collection, which in turn expedites model convergence. We conducted extensive simulations across various scenarios, benchmarking our DPRL algorithm against the state-of-the-art navigation algorithms. The results consistently demonstrate the superior performance of our algorithm in terms of flight efficiency, robustness and overall success rate.

Paper Structure

This paper contains 16 sections, 1 equation, 28 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision-based UAV Navigation
  4. Strategies to Accelerate DRL Convergence
  5. Methodology
  6. Problem Formulation
  7. Privileged Reinforcement Learning
  8. Reward Function Design
  9. Policy Learning with Privileged Information
  10. Neural Network Architecture
  11. Multi-agent Exploration Strategy
  12. Experiments and Results
  13. Experiments Setup
  14. Comparison Experiment
  15. Ablation Experiment
  16. ...and 1 more sections

Figures (28)

  • Figure 1: The DPRL Framework for UAV Navigation.
  • Figure 3: TD3 Framework within the POMDP Model.
  • Figure 4: Architecture of Actor and Critic Network.
  • Figure : (a)
  • Figure : (a)
  • ...and 23 more figures