Optimal Actuator Attacks on Autonomous Vehicles Using Reinforcement Learning

TL;DR

This paper addresses the vulnerability of autonomous-vehicle actuators to stealthy integrity attacks and critiques existing RL-based defenses. It proposes a reinforcement-learning framework that designs optimal actuator FDI attacks by maximizing the objective $J=\lim_{N\to\infty} \frac{1}{N}\sum_{k=1}^N (J_t-J_e+J_s)$, where $J_t$ is tracking error, $J_e$ is energy, and $J_s$ enforces stealth against a detector. The attacker is trained via both PPO and SAC, with PPO offering better stability and performance in the nonlinear AV setting, and results show improved stealth (detector recall) alongside larger tracking deviations and energy usage compared to prior work. The study uses EKF state estimation, a dynamic $\chi^2$ detector, and OpenAI Gym simulations to demonstrate the method’s effectiveness, while highlighting limitations in current secure controllers and outlining directions for robust defense and real-world deployment.

Abstract

With the increasing prevalence of autonomous vehicles (AVs), their vulnerability to various types of attacks has grown, presenting significant security challenges. In this paper, we propose a reinforcement learning (RL)-based approach for designing optimal stealthy integrity attacks on AV actuators. We also analyze the limitations of state-of-the-art RL-based secure controllers developed to counter such attacks. Through extensive simulation experiments, we demonstrate the effectiveness and efficiency of our proposed method.

Figures (5)

• Figure 1: AV attacked while executing a circular trajectory.
• Figure 2: Training results for two RL algorithms.
• Figure 3: Dynamic $\chi ^{2}$ attack detector operation process.
• Figure 4: Trajectory visualization in training phase.
• Figure 5: Trajectory visualization in inference phase.