Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VCAT: Vulnerability-aware and Curiosity-driven Adversarial Training for Enhancing Autonomous Vehicle Robustness

Xuan Cai, Zhiyong Cui, Xuesong Bai, Ruimin Ke, Zhenshu Ma, Haiyang Yu, Yilong Ren

TL;DR

Autonomous vehicles confront safety-critical risks in complex traffic, and existing adversarial training often fails to exploit the victim's intrinsic vulnerabilities or to explore the attacker’s policy space adequately. The authors propose VCAT, a vulnerability-aware and curiosity-driven adversarial training framework that combines a Victim Value Approximation Network to reveal vulnerabilities with Random Network Distillation to drive intrinsic exploration, all trained within a PPO-based attacker. The method alternates between an adversarial attack phase and an adversarial defense phase, enabling the victim to robustly counter pretrained attackers. In highway-env simulations, VCAT improves robust control and reduces crashes more effectively than conventional adversarial training and other RL baselines, particularly in rare edge-case scenarios. These findings offer a scalable approach to hardening AVs against sophisticated, sparsely rewarded adversaries, with available code to enable replication and extension.

Abstract

Autonomous vehicles (AVs) face significant threats to their safe operation in complex traffic environments. Adversarial training has emerged as an effective method of enabling AVs to preemptively fortify their robustness against malicious attacks. Train an attacker using an adversarial policy, allowing the AV to learn robust driving through interaction with this attacker. However, adversarial policies in existing methodologies often get stuck in a loop of overexploiting established vulnerabilities, resulting in poor improvement for AVs. To overcome the limitations, we introduce a pioneering framework termed Vulnerability-aware and Curiosity-driven Adversarial Training (VCAT). Specifically, during the traffic vehicle attacker training phase, a surrogate network is employed to fit the value function of the AV victim, providing dense information about the victim's inherent vulnerabilities. Subsequently, random network distillation is used to characterize the novelty of the environment, constructing an intrinsic reward to guide the attacker in exploring unexplored territories. In the victim defense training phase, the AV is trained in critical scenarios in which the pretrained attacker is positioned around the victim to generate attack behaviors. Experimental results revealed that the training methodology provided by VCAT significantly improved the robust control capabilities of learning-based AVs, outperforming both conventional training modalities and alternative reinforcement learning counterparts, with a marked reduction in crash rates. The code is available at https://github.com/caixxuan/VCAT.

VCAT: Vulnerability-aware and Curiosity-driven Adversarial Training for Enhancing Autonomous Vehicle Robustness

TL;DR

Autonomous vehicles confront safety-critical risks in complex traffic, and existing adversarial training often fails to exploit the victim's intrinsic vulnerabilities or to explore the attacker’s policy space adequately. The authors propose VCAT, a vulnerability-aware and curiosity-driven adversarial training framework that combines a Victim Value Approximation Network to reveal vulnerabilities with Random Network Distillation to drive intrinsic exploration, all trained within a PPO-based attacker. The method alternates between an adversarial attack phase and an adversarial defense phase, enabling the victim to robustly counter pretrained attackers. In highway-env simulations, VCAT improves robust control and reduces crashes more effectively than conventional adversarial training and other RL baselines, particularly in rare edge-case scenarios. These findings offer a scalable approach to hardening AVs against sophisticated, sparsely rewarded adversaries, with available code to enable replication and extension.

Abstract

Autonomous vehicles (AVs) face significant threats to their safe operation in complex traffic environments. Adversarial training has emerged as an effective method of enabling AVs to preemptively fortify their robustness against malicious attacks. Train an attacker using an adversarial policy, allowing the AV to learn robust driving through interaction with this attacker. However, adversarial policies in existing methodologies often get stuck in a loop of overexploiting established vulnerabilities, resulting in poor improvement for AVs. To overcome the limitations, we introduce a pioneering framework termed Vulnerability-aware and Curiosity-driven Adversarial Training (VCAT). Specifically, during the traffic vehicle attacker training phase, a surrogate network is employed to fit the value function of the AV victim, providing dense information about the victim's inherent vulnerabilities. Subsequently, random network distillation is used to characterize the novelty of the environment, constructing an intrinsic reward to guide the attacker in exploring unexplored territories. In the victim defense training phase, the AV is trained in critical scenarios in which the pretrained attacker is positioned around the victim to generate attack behaviors. Experimental results revealed that the training methodology provided by VCAT significantly improved the robust control capabilities of learning-based AVs, outperforming both conventional training modalities and alternative reinforcement learning counterparts, with a marked reduction in crash rates. The code is available at https://github.com/caixxuan/VCAT.
Paper Structure (25 sections, 6 equations, 5 figures, 3 tables, 1 algorithm)

This paper contains 25 sections, 6 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Problems and Challenges
  3. Main Contribution
  4. Construction
  5. RELATED WORKS
  6. DRL-powered Attack
  7. Adversarial Training
  8. PROPOSED METHOD
  9. Overview of the VCAT Framework
  10. Adversarial attack
  11. Victim Value Approximation Network
  12. Curiosity-Driven Exploration
  13. Attacker Policy Training
  14. Adversarial Defense
  15. EXPERIMENT
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Overview of the proposed VCAT framework. VCAT is divided into two stages: adversarial attack, enclosed by the left dashed box, and adversarial defense, enclosed by the right one. The snowflake pattern indicates that the neural network parameters are frozen, while the flame pattern indicates that the parameters can be adjusted for learning. The horizontal line with the reverse triangle arrow represents the minimum Euclidean distance between the two. The circled cross signifies data concatenation.
  • Figure 2: Illustration for the setup of the three scenarios. The trajectory of the AV (victim) is represented by the red line, while the trajectory of the traffic vehicle (attacker) is represented by the black line. The scenarios are (a) # NoSignalJunctionCrossingRoute (# NSJCR), (b) SignalizedJunctionRightTurn (# SJRT), and (c) SignalizedJunctionLeftTurn (# SJLT), respectively. The abbreviations are used hereafter.
  • Figure 3: Crash rate in the adversarial attack training with different methods. The orange "V"-shaped arrows highlight the decline-rise process experienced by the proposed method.
  • Figure 4: t-SNE visualization of the victim (target AV) in the attack training under the three scenarios. The size of the coordinate axis is consistent for each scenario.
  • Figure 5: Number of crashes ($\uparrow$) in different types under the three scenarios.