Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RL-Based Method for Benchmarking the Adversarial Resilience and Robustness of Deep Reinforcement Learning Policies

Vahid Behzadan, William Hsu

TL;DR

This paper presents an approach for the disentanglement of vulnerabilities caused by representation learning of DRL agents from those that stem from the sensitivity of the DRL policies to distributional shifts in state transitions and proposes two RL-based techniques for quantitative benchmarking of adversarial resilience and robustness in D RL policies against perturbations of state transitions.

Abstract

This paper investigates the resilience and robustness of Deep Reinforcement Learning (DRL) policies to adversarial perturbations in the state space. We first present an approach for the disentanglement of vulnerabilities caused by representation learning of DRL agents from those that stem from the sensitivity of the DRL policies to distributional shifts in state transitions. Building on this approach, we propose two RL-based techniques for quantitative benchmarking of adversarial resilience and robustness in DRL policies against perturbations of state transitions. We demonstrate the feasibility of our proposals through experimental evaluation of resilience and robustness in DQN, A2C, and PPO2 policies trained in the Cartpole environment.

RL-Based Method for Benchmarking the Adversarial Resilience and Robustness of Deep Reinforcement Learning Policies

TL;DR

This paper presents an approach for the disentanglement of vulnerabilities caused by representation learning of DRL agents from those that stem from the sensitivity of the DRL policies to distributional shifts in state transitions and proposes two RL-based techniques for quantitative benchmarking of adversarial resilience and robustness in D RL policies against perturbations of state transitions.

Abstract

This paper investigates the resilience and robustness of Deep Reinforcement Learning (DRL) policies to adversarial perturbations in the state space. We first present an approach for the disentanglement of vulnerabilities caused by representation learning of DRL agents from those that stem from the sensitivity of the DRL policies to distributional shifts in state transitions. Building on this approach, we propose two RL-based techniques for quantitative benchmarking of adversarial resilience and robustness in DRL policies against perturbations of state transitions. We demonstrate the feasibility of our proposals through experimental evaluation of resilience and robustness in DQN, A2C, and PPO2 policies trained in the Cartpole environment.

Paper Structure

This paper contains 13 sections, 3 equations, 12 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Benchmarking of Test-Time Resilience
  4. Benchmarking of Test-Time Robustness
  5. Experiment Setup
  6. Results
  7. Resilience Benchmarks
  8. Training Results:
  9. Test-Time Step-Perturbation Distribution:
  10. Robustness Benchmarks
  11. Case 1: $\delta_{max} = 10$:
  12. Case 2: $\delta_{max} = 5$:
  13. Conclusion

Figures (12)

  • Figure 1: Adversarial Training Progress for Resilience Benchmarking of the DQN Policy
  • Figure 2: Adversarial Training Progress for Resilience Benchmarking of the A2C Policy
  • Figure 3: Adversarial Training Progress for Resilience Benchmarking of the PPO2 Policy
  • Figure 4: Perturbation Count Per Episodic TimeStep in 100 Runs Targeting DQN Policy
  • Figure 5: Perturbation Count Per Episodic TimeStep in 100 Runs Targeting A2C Policy
  • ...and 7 more figures