Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intercepting Unauthorized Aerial Robots in Controlled Airspace Using Reinforcement Learning

Francisco Giral, Ignacio Gómez, Soledad Le Clainche

TL;DR

The paper tackles intercepting unauthorized UAVs in controlled airspace under dynamic evasion and environmental disturbances. It compares a model-based RL approach (DreamerV3) with two model-free methods (TQC and SAC) using a high-fidelity JSBSim environment that simulates both pursuer and evader dynamics. Key contributions include a full MDPlabeled training framework, robustness validation against unseen evaders and perturbations, and empirical evidence that DreamerV3 generalizes well and remains stable under perturbations while SAC lags. The work demonstrates the viability of model-based RL for adaptive UAV interception and informs design choices for safe and efficient airspace management in urban and critical-infrastructure contexts.

Abstract

The proliferation of unmanned aerial vehicles (UAVs) in controlled airspace presents significant risks, including potential collisions, disruptions to air traffic, and security threats. Ensuring the safe and efficient operation of airspace, particularly in urban environments and near critical infrastructure, necessitates effective methods to intercept unauthorized or non-cooperative UAVs. This work addresses the critical need for robust, adaptive systems capable of managing such threats through the use of Reinforcement Learning (RL). We present a novel approach utilizing RL to train fixed-wing UAV pursuer agents for intercepting dynamic evader targets. Our methodology explores both model-based and model-free RL algorithms, specifically DreamerV3, Truncated Quantile Critics (TQC), and Soft Actor-Critic (SAC). The training and evaluation of these algorithms were conducted under diverse scenarios, including unseen evasion strategies and environmental perturbations. Our approach leverages high-fidelity flight dynamics simulations to create realistic training environments. This research underscores the importance of developing intelligent, adaptive control systems for UAV interception, significantly contributing to the advancement of secure and efficient airspace management. It demonstrates the potential of RL to train systems capable of autonomously achieving these critical tasks.

Intercepting Unauthorized Aerial Robots in Controlled Airspace Using Reinforcement Learning

TL;DR

The paper tackles intercepting unauthorized UAVs in controlled airspace under dynamic evasion and environmental disturbances. It compares a model-based RL approach (DreamerV3) with two model-free methods (TQC and SAC) using a high-fidelity JSBSim environment that simulates both pursuer and evader dynamics. Key contributions include a full MDPlabeled training framework, robustness validation against unseen evaders and perturbations, and empirical evidence that DreamerV3 generalizes well and remains stable under perturbations while SAC lags. The work demonstrates the viability of model-based RL for adaptive UAV interception and informs design choices for safe and efficient airspace management in urban and critical-infrastructure contexts.

Abstract

The proliferation of unmanned aerial vehicles (UAVs) in controlled airspace presents significant risks, including potential collisions, disruptions to air traffic, and security threats. Ensuring the safe and efficient operation of airspace, particularly in urban environments and near critical infrastructure, necessitates effective methods to intercept unauthorized or non-cooperative UAVs. This work addresses the critical need for robust, adaptive systems capable of managing such threats through the use of Reinforcement Learning (RL). We present a novel approach utilizing RL to train fixed-wing UAV pursuer agents for intercepting dynamic evader targets. Our methodology explores both model-based and model-free RL algorithms, specifically DreamerV3, Truncated Quantile Critics (TQC), and Soft Actor-Critic (SAC). The training and evaluation of these algorithms were conducted under diverse scenarios, including unseen evasion strategies and environmental perturbations. Our approach leverages high-fidelity flight dynamics simulations to create realistic training environments. This research underscores the importance of developing intelligent, adaptive control systems for UAV interception, significantly contributing to the advancement of secure and efficient airspace management. It demonstrates the potential of RL to train systems capable of autonomously achieving these critical tasks.
Paper Structure (18 sections, 16 equations, 11 figures, 7 tables)

This paper contains 18 sections, 16 equations, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation
  4. Simulation Environment
  5. Reinforcement Learning Algorithms Overview
  6. Model-free RL
  7. Model-based RL
  8. Markov Decision Process Formulation of the Problem
  9. State Space Formulation
  10. Action Space Formulation
  11. Reward Function Definition
  12. Training Framework
  13. Validation Scenarios
  14. Results
  15. Training Results
  16. ...and 3 more sections

Figures (11)

  • Figure 1: Diagram illustrating the problem setup and the Reinforcement Learning (RL) training framework. The pursuer agent, controlled by the RL policy, interacts with the environment that simulates the flight dynamics of both the pursuer and the evader UAVs. The evader is controlled by a setpoint tracking controller, enabling various evasion strategies. Actions flow are displayed in blue, States and Reward flows are displayed in red and purple respectively, showing the interaction between agent and environment.
  • Figure 2: Diagram of the Dreamer algorithm. The agent learns a world model using observed data, then uses this model for imagination-based training of the actor and critic networks. Finally, the trained policy is applied in the real environment.
  • Figure 3: Diagram showing the Aspect Angle (AA) and Antenna Train Angle (ATA) between the pursuer and target UAVs.
  • Figure 4: Reward shaping function for different values of the shaping parameter $k$.
  • Figure 5: Example of setpoint controller policy usage for the evader UAV tracking reference values for altitude, heading, and airspeed.
  • ...and 6 more figures