Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks

TL;DR

This work demonstrates that Deep Q-Networks are susceptible to adversarial input perturbations and that adversarial examples can transfer between DQN models. It introduces a policy-induction attack exploiting this transferability, framed as initialization and exploitation phases that craft perturbations to steer learning toward an adversary’s policy. Empirical results on Pong show successful adversarial manipulation and degraded rewards, with transferability confirmed across different networks. The study highlights the vulnerability of DRL systems and underscores the need for robust defenses and analytical guarantees for policy-driven agents in safety-critical settings.

Abstract

Deep learning classifiers are known to be inherently vulnerable to manipulation by intentionally perturbed inputs, named adversarial examples. In this work, we establish that reinforcement learning techniques based on Deep Q-Networks (DQNs) are also vulnerable to adversarial input perturbations, and verify the transferability of adversarial examples across different DQN models. Furthermore, we present a novel class of attacks based on this vulnerability that enable policy manipulation and induction in the learning process of DQNs. We propose an attack mechanism that exploits the transferability of adversarial examples to implement policy induction attacks on DQNs, and demonstrate its efficacy and impact through experimental study of a game-learning scenario.

Figures (6)

• Figure 1: DQN architecture for end-to-end learning of Atari 2600 game plays
• Figure 2: Exploitation cycle of policy induction attack
• Figure 3: Game of Pong
• Figure 4: Success rate of crafting adversarial examples for DQN
• Figure 5: Transferability of adversarial examples in DQN