Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Tracking

Ala Souissi, Hajer Fradi, Panagiotis Papadakis

TL;DR

This paper proposes an end-to-end deep-reinforcement learning (DRL) framework that maps raw sensor data from event streams directly to control actions for the UAV, and demonstrates the effectiveness of the approach through experiments in challenging scenarios, including fast-moving targets and changing lighting conditions.

Abstract

In this paper, we present our proposed approach for active tracking to increase the autonomy of Unmanned Aerial Vehicles (UAVs) using event cameras, low-energy imaging sensors that offer significant advantages in speed and dynamic range. The proposed tracking controller is designed to respond to visual feedback from the mounted event sensor, adjusting the drone movements to follow the target. To leverage the full motion capabilities of a quadrotor and the unique properties of event sensors, we propose an end-to-end deep-reinforcement learning (DRL) framework that maps raw sensor data from event streams directly to control actions for the UAV. To learn an optimal policy under highly variable and challenging conditions, we opt for a simulation environment with domain randomization for effective transfer to real-world environments. We demonstrate the effectiveness of our approach through experiments in challenging scenarios, including fast-moving targets and changing lighting conditions, which result in improved generalization capabilities.

Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Tracking

TL;DR

This paper proposes an end-to-end deep-reinforcement learning (DRL) framework that maps raw sensor data from event streams directly to control actions for the UAV, and demonstrates the effectiveness of the approach through experiments in challenging scenarios, including fast-moving targets and changing lighting conditions.

Abstract

In this paper, we present our proposed approach for active tracking to increase the autonomy of Unmanned Aerial Vehicles (UAVs) using event cameras, low-energy imaging sensors that offer significant advantages in speed and dynamic range. The proposed tracking controller is designed to respond to visual feedback from the mounted event sensor, adjusting the drone movements to follow the target. To leverage the full motion capabilities of a quadrotor and the unique properties of event sensors, we propose an end-to-end deep-reinforcement learning (DRL) framework that maps raw sensor data from event streams directly to control actions for the UAV. To learn an optimal policy under highly variable and challenging conditions, we opt for a simulation environment with domain randomization for effective transfer to real-world environments. We demonstrate the effectiveness of our approach through experiments in challenging scenarios, including fast-moving targets and changing lighting conditions, which result in improved generalization capabilities.

Paper Structure

This paper contains 23 sections, 11 equations, 7 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. Related work
  3. Visual Tracking
  4. Active Drone Tracking
  5. Preliminary definitions
  6. Dynamic Quadrocopter Model
  7. Reinforcement Learning-based Tracking
  8. Method
  9. Asymmetric Soft Actor-Critic Framework
  10. Asymmetric Actor-Critic
  11. Reward shaping
  12. End-to-End Event-based Tracking Architecture
  13. Event Data Processing
  14. Neural Networks of Actor and Critic
  15. Experiments and Results
  16. ...and 8 more sections

Figures (7)

  • Figure 1: DRL controller using event streams vs. classic controller for UAV active tracking.
  • Figure 2: The flowchart of Asymmetric Soft Actor-Critic (ASAC) learning framework: the actor and critic networks work together to optimize the policy.
  • Figure 3: Overview of the proposed end-to-end architecture processing events for UAV active tracking.
  • Figure 4: Examples of training box environments with varying textures (first row) and DARPA environments used for evaluation (second row).
  • Figure 5: Learning plots of the detection-based (baseline), end-to-end RGB-based (E2E RGB), and end-to-end event-based (E2E event) approaches.
  • ...and 2 more figures