Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GARL: Genetic Algorithm-Augmented Reinforcement Learning to Detect Violations in Marker-Based Autonomous Landing Systems

Linfeng Liang, Yao Deng, Kye Morton, Valtteri Kallinen, Alice James, Avishkar Seth, Endrowednes Kuantama, Subhas Mukhopadhyay, Richard Han, Xi Zheng

TL;DR

GARL, a framework combining a genetic algorithm (GA) and reinforcement learning (RL) for efficient generation of diverse and real landing system failures within a practical budget, is introduced, pioneering the integration of offline and online testing strategies for autonomous systems.

Abstract

Automated Uncrewed Aerial Vehicle (UAV) landing is crucial for autonomous UAV services such as monitoring, surveying, and package delivery. It involves detecting landing targets, perceiving obstacles, planning collision-free paths, and controlling UAV movements for safe landing. Failures can lead to significant losses, necessitating rigorous simulation-based testing for safety. Traditional offline testing methods, limited to static environments and predefined trajectories, may miss violation cases caused by dynamic objects like people and animals. Conversely, online testing methods require extensive training time, which is impractical with limited budgets. To address these issues, we introduce GARL, a framework combining a genetic algorithm (GA) and reinforcement learning (RL) for efficient generation of diverse and real landing system failures within a practical budget. GARL employs GA for exploring various environment setups offline, reducing the complexity of RL's online testing in simulating challenging landing scenarios. Our approach outperforms existing methods by up to 18.35% in violation rate and 58% in diversity metric. We validate most discovered violation types with real-world UAV tests, pioneering the integration of offline and online testing strategies for autonomous systems. This method opens new research directions for online testing, with our code and supplementary material available at https://github.com/lfeng0722/drone_testing/.

GARL: Genetic Algorithm-Augmented Reinforcement Learning to Detect Violations in Marker-Based Autonomous Landing Systems

TL;DR

GARL, a framework combining a genetic algorithm (GA) and reinforcement learning (RL) for efficient generation of diverse and real landing system failures within a practical budget, is introduced, pioneering the integration of offline and online testing strategies for autonomous systems.

Abstract

Automated Uncrewed Aerial Vehicle (UAV) landing is crucial for autonomous UAV services such as monitoring, surveying, and package delivery. It involves detecting landing targets, perceiving obstacles, planning collision-free paths, and controlling UAV movements for safe landing. Failures can lead to significant losses, necessitating rigorous simulation-based testing for safety. Traditional offline testing methods, limited to static environments and predefined trajectories, may miss violation cases caused by dynamic objects like people and animals. Conversely, online testing methods require extensive training time, which is impractical with limited budgets. To address these issues, we introduce GARL, a framework combining a genetic algorithm (GA) and reinforcement learning (RL) for efficient generation of diverse and real landing system failures within a practical budget. GARL employs GA for exploring various environment setups offline, reducing the complexity of RL's online testing in simulating challenging landing scenarios. Our approach outperforms existing methods by up to 18.35% in violation rate and 58% in diversity metric. We validate most discovered violation types with real-world UAV tests, pioneering the integration of offline and online testing strategies for autonomous systems. This method opens new research directions for online testing, with our code and supplementary material available at https://github.com/lfeng0722/drone_testing/.
Paper Structure (36 sections, 6 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 36 sections, 6 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. GA-based Test Generation
  4. RL-based Test Generation
  5. Combining GA and RL
  6. Method
  7. Overview
  8. Modeling Test Scenario
  9. Genetic Algorithm - Offline
  10. Multi-Objective Fitness Function
  11. Variation Operation
  12. Reinforcement Learning - Online
  13. Build Surrogate RL Training Environment
  14. Define Reward Function
  15. Experiment
  16. ...and 21 more sections

Figures (9)

  • Figure 1: A high-level overview of GARL framework, including the simulator and the test engine.
  • Figure 2: Workflow of GARL
  • Figure 3: The Chromosome Representation of Test Case
  • Figure 4: Process of Nuclear Gene Crossover
  • Figure 5: Example of test map in real-world (First row) and AirSim (Second row).
  • ...and 4 more figures