Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cooperative Search and Track of Rogue Drones using Multiagent Reinforcement Learning

Panayiota Valianti, Kleanthis Malialis, Panayiotis Kolios, Georgios Ellinas

TL;DR

The paper tackles the problem of locating and tracking rogue drones over sensitive facilities using a team of pursuer UAVs with limited sensing. It models joint search-and-track as a multiagent MDP and introduces a cooperative Q-learning framework with difference rewards to address credit assignment, aiming to maximize $F = \sum_{t=1}^{T} \sum_{i \in I} N^i_t$. Empirical results show that difference rewards yield faster learning and higher final performance than a global reward, especially as the number of agents grows, approaching near-optimal coverage when target trajectories are known. The work provides a scalable, model-free solution for coordinated search-and-track and outlines a path toward integrated search-track-jam interception and real-world field trials.

Abstract

This work considers the problem of intercepting rogue drones targeting sensitive critical infrastructure facilities. While current interception technologies focus mainly on the jamming/spoofing tasks, the challenges of effectively locating and tracking rogue drones have not received adequate attention. Solving this problem and integrating with recently proposed interception techniques will enable a holistic system that can reliably detect, track, and neutralize rogue drones. Specifically, this work considers a team of pursuer UAVs that can search, detect, and track multiple rogue drones over a sensitive facility. The joint search and track problem is addressed through a novel multiagent reinforcement learning scheme to optimize the agent mobility control actions that maximize the number of rogue drones detected and tracked. The performance of the proposed system is investigated under realistic settings through extensive simulation experiments with varying number of agents demonstrating both its performance and scalability.

Cooperative Search and Track of Rogue Drones using Multiagent Reinforcement Learning

TL;DR

The paper tackles the problem of locating and tracking rogue drones over sensitive facilities using a team of pursuer UAVs with limited sensing. It models joint search-and-track as a multiagent MDP and introduces a cooperative Q-learning framework with difference rewards to address credit assignment, aiming to maximize . Empirical results show that difference rewards yield faster learning and higher final performance than a global reward, especially as the number of agents grows, approaching near-optimal coverage when target trajectories are known. The work provides a scalable, model-free solution for coordinated search-and-track and outlines a path toward integrated search-track-jam interception and real-world field trials.

Abstract

This work considers the problem of intercepting rogue drones targeting sensitive critical infrastructure facilities. While current interception technologies focus mainly on the jamming/spoofing tasks, the challenges of effectively locating and tracking rogue drones have not received adequate attention. Solving this problem and integrating with recently proposed interception techniques will enable a holistic system that can reliably detect, track, and neutralize rogue drones. Specifically, this work considers a team of pursuer UAVs that can search, detect, and track multiple rogue drones over a sensitive facility. The joint search and track problem is addressed through a novel multiagent reinforcement learning scheme to optimize the agent mobility control actions that maximize the number of rogue drones detected and tracked. The performance of the proposed system is investigated under realistic settings through extensive simulation experiments with varying number of agents demonstrating both its performance and scalability.
Paper Structure (15 sections, 8 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 15 sections, 8 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. State of the Art
  4. Background on Reinforcement Learning
  5. System Model
  6. Target Dynamics
  7. Agent Dynamics
  8. Agent Sensing Model
  9. Problem Formulation
  10. Proposed Approach
  11. Proposed Cooperative Q-learning Algorithm
  12. Performance Evaluation
  13. Simulation Setup
  14. Simulation Results
  15. Conclusions

Figures (4)

  • Figure 1: Multiple UAV agents searching and tracking multiple rogue drones. Each agent detects targets within its detection radius in each of its four sensing areas.
  • Figure 2: Training results for different number of agents and reward functions.
  • Figure 3: Maneuvers of $4$ search-and-track agents in a simulated scenario with $2$ targets, during $3$ time periods (agent start and stop positions are marked with $\star$ and $\blacklozenge$, respectively, and target start and stop positions are marked with $\bullet$ and $\times$, respectively).
  • Figure 4: Optimal (P1), and learned ($G$, $D$) agent trajectories (covering the area under consideration) averaged over all training episodes (visits per square meter) depicted for the case of $4$ pursuing agents.