Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable AI Based Diagnosis of Poisoning Attacks in Evolutionary Swarms

Mehrdad Asadi, Roxana Rădulescu, Ann Nowé

TL;DR

PADEX addresses data-poisoning risk in autonomous swarm coordination by integrating an evolutionary-game swarm model with ML-based surrogate modeling and SHAP-based explainability to diagnose misbehavior. The instantiation on multi-drone cooperative sampling demonstrates that poisoning above $10\%$ degrades coordination and performance (e.g., accuracy from $91\%$ to $63\%$), while SHAP signatures enable early detection of deviations. This framework enables severity characterization and proactive diagnosis, contributing to resilience and safety for real-world swarm deployments.

Abstract

Swarming systems, such as for example multi-drone networks, excel at cooperative tasks like monitoring, surveillance, or disaster assistance in critical environments, where autonomous agents make decentralized decisions in order to fulfill team-level objectives in a robust and efficient manner. Unfortunately, team-level coordinated strategies in the wild are vulnerable to data poisoning attacks, resulting in either inaccurate coordination or adversarial behavior among the agents. To address this challenge, we contribute a framework that investigates the effects of such data poisoning attacks, using explainable AI methods. We model the interaction among agents using evolutionary intelligence, where an optimal coalition strategically emerges to perform coordinated tasks. Then, through a rigorous evaluation, the swarm model is systematically poisoned using data manipulation attacks. We showcase the applicability of explainable AI methods to quantify the effects of poisoning on the team strategy and extract footprint characterizations that enable diagnosing. Our findings indicate that when the model is poisoned above 10%, non-optimal strategies resulting in inefficient cooperation can be identified.

Explainable AI Based Diagnosis of Poisoning Attacks in Evolutionary Swarms

TL;DR

PADEX addresses data-poisoning risk in autonomous swarm coordination by integrating an evolutionary-game swarm model with ML-based surrogate modeling and SHAP-based explainability to diagnose misbehavior. The instantiation on multi-drone cooperative sampling demonstrates that poisoning above degrades coordination and performance (e.g., accuracy from to ), while SHAP signatures enable early detection of deviations. This framework enables severity characterization and proactive diagnosis, contributing to resilience and safety for real-world swarm deployments.

Abstract

Swarming systems, such as for example multi-drone networks, excel at cooperative tasks like monitoring, surveillance, or disaster assistance in critical environments, where autonomous agents make decentralized decisions in order to fulfill team-level objectives in a robust and efficient manner. Unfortunately, team-level coordinated strategies in the wild are vulnerable to data poisoning attacks, resulting in either inaccurate coordination or adversarial behavior among the agents. To address this challenge, we contribute a framework that investigates the effects of such data poisoning attacks, using explainable AI methods. We model the interaction among agents using evolutionary intelligence, where an optimal coalition strategically emerges to perform coordinated tasks. Then, through a rigorous evaluation, the swarm model is systematically poisoned using data manipulation attacks. We showcase the applicability of explainable AI methods to quantify the effects of poisoning on the team strategy and extract footprint characterizations that enable diagnosing. Our findings indicate that when the model is poisoned above 10%, non-optimal strategies resulting in inefficient cooperation can be identified.
Paper Structure (11 sections, 4 figures)

This paper contains 11 sections, 4 figures.

Table of Contents

  1. Introduction
  2. The PADEX Diagnosis Framework
  3. PADEX Instantiation
  4. Evolutionary Dynamics of Swarm
  5. Assumptions and Threat Modeling
  6. Feature Importance Analysis
  7. Experimental Setup
  8. Results
  9. XAI-based Feature Quantification
  10. Related Work
  11. Conclusion and Future Work

Figures (4)

  • Figure 1: Overview of the modules and pipeline of the PADEX framework
  • Figure 2: An instantiation of PADEX; a) Evolutionary Game Modeling b) Feature Perturbation and Model Training (Random Forest); and c) SHAP-based diagnosis
  • Figure 3: Distribution of predicted strategies for the testing data
  • Figure 4: Effects of features on the output of 100 samples. The features are agents' positions (x,y) in a grid-like environment. Blue band shows positive contribution, and pink color shows negative impact on prediction.