Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adversarial Attacks on Reinforcement Learning Agents for Command and Control

Ahaan Dabholkar, James Z. Hare, Mark Mittrick, John Richardson, Nicholas Waytowich, Priya Narayanan, Saurabh Bagchi

TL;DR

This work analyzes the vulnerability of reinforcement learning agents used for command-and-control in battlefield-like simulations. By training C2 policies with A3C and PPO on two custom StarCraft II scenarios and applying FGSM-based inference-time perturbations to observations, the study shows that small perturbations can markedly degrade performance and alter strategic behavior. The findings reveal a fragility in vanilla RL approaches for critical domains and highlight the need for robust training, detection, and defense mechanisms to enable reliable, adversary-resilient C2 decision-support. The work provides actionable insights into which input channels (notably the visual screen) drive attacks and how training choices influence robustness, with implications for real-world defense and security of autonomous decision systems.

Abstract

Given the recent impact of Deep Reinforcement Learning in training agents to win complex games like StarCraft and DoTA(Defense Of The Ancients) - there has been a surge in research for exploiting learning based techniques for professional wargaming, battlefield simulation and modeling. Real time strategy games and simulators have become a valuable resource for operational planning and military research. However, recent work has shown that such learning based approaches are highly susceptible to adversarial perturbations. In this paper, we investigate the robustness of an agent trained for a Command and Control task in an environment that is controlled by an active adversary. The C2 agent is trained on custom StarCraft II maps using the state of the art RL algorithms - A3C and PPO. We empirically show that an agent trained using these algorithms is highly susceptible to noise injected by the adversary and investigate the effects these perturbations have on the performance of the trained agent. Our work highlights the urgent need to develop more robust training algorithms especially for critical arenas like the battlefield.

Adversarial Attacks on Reinforcement Learning Agents for Command and Control

TL;DR

This work analyzes the vulnerability of reinforcement learning agents used for command-and-control in battlefield-like simulations. By training C2 policies with A3C and PPO on two custom StarCraft II scenarios and applying FGSM-based inference-time perturbations to observations, the study shows that small perturbations can markedly degrade performance and alter strategic behavior. The findings reveal a fragility in vanilla RL approaches for critical domains and highlight the need for robust training, detection, and defense mechanisms to enable reliable, adversary-resilient C2 decision-support. The work provides actionable insights into which input channels (notably the visual screen) drive attacks and how training choices influence robustness, with implications for real-world defense and security of autonomous decision systems.

Abstract

Given the recent impact of Deep Reinforcement Learning in training agents to win complex games like StarCraft and DoTA(Defense Of The Ancients) - there has been a surge in research for exploiting learning based techniques for professional wargaming, battlefield simulation and modeling. Real time strategy games and simulators have become a valuable resource for operational planning and military research. However, recent work has shown that such learning based approaches are highly susceptible to adversarial perturbations. In this paper, we investigate the robustness of an agent trained for a Command and Control task in an environment that is controlled by an active adversary. The C2 agent is trained on custom StarCraft II maps using the state of the art RL algorithms - A3C and PPO. We empirically show that an agent trained using these algorithms is highly susceptible to noise injected by the adversary and investigate the effects these perturbations have on the performance of the trained agent. Our work highlights the urgent need to develop more robust training algorithms especially for critical arenas like the battlefield.
Paper Structure (27 sections, 2 equations, 18 figures)

This paper contains 27 sections, 2 equations, 18 figures.

Table of Contents

  1. Introduction
  2. Background
  3. RL for Command and Control (C2)
  4. StarCraft II C2 Environment
  5. Custom StarCraft II Scenarios
  6. Reinforcement Learning Algorithms
  7. RL Environment
  8. State and Action Space
  9. Reward Structure
  10. RL Agent Description
  11. The Policy Network
  12. Adversarial Attacks on RL agents
  13. Adversarial Attacks on Image Classifiers
  14. Fast Gradient Sign Method (FGSM) goodfellow2014explaining.
  15. Inference Time Attacks on Policies
  16. ...and 12 more sections

Figures (18)

  • Figure 1: Robustness Evaluation Methodology: The figure shows the difference between a benign (top) and malicious (bottom) environment at timestep $t$. Observations at $t-1$ are input to a C2 agent that has been pretrained in a benign environment. The agent samples a suboptimal action as a result of the injected adversarial perturbations (orange) in the input which eventually leads to a loss for the BlueForce.
  • Figure 2: TigerClaw Scenario
  • Figure 3: TigerClaw: (Right) The geographical map of the scenario and (Left) the correspondingly designed map in StarCraft
  • Figure 4: NTC: (Right) The geographical map of the scenario and (Left) the correspondingly designed map in StarCraft
  • Figure 5: C2 Policy Network: Compuational graph of the policy network of our C2 agent. The inputs and outputs are shown in blue and yellow respectively. Shaded rectangles represent the concatenate operation. Conv2D and FC layers are ReLU activated.
  • ...and 13 more figures