Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllers

TL;DR

This work addresses the vulnerability of learning-based quadrupedal locomotion controllers by introducing sequential adversarial attacks that reveal failure modes not captured by standard robustness measures. It develops a Lipschitz-regularized RL framework to learn time-series adversaries across observation, command, and perturbation spaces, and validates findings on both a simulated platform and real hardware, including the DARPA SubT-winning policy. The authors show that domain randomization is insufficient alone for robustness, and that multi-modal, terrain-aware adversaries combined with adversarial finetuning significantly improve safety and reliability, effectively serving as a robustness diagnostic and enhancement tool. The approach extends to other controllers (e.g., MPC) and offers practical insight for safety verification and deployment of neural locomotion policies in complex environments.

Abstract

Legged locomotion has recently achieved remarkable success with the progress of machine learning techniques, especially deep reinforcement learning (RL). Controllers employing neural networks have demonstrated empirical and qualitative robustness against real-world uncertainties, including sensor noise and external perturbations. However, formally investigating the vulnerabilities of these locomotion controllers remains a challenge. This difficulty arises from the requirement to pinpoint vulnerabilities across a long-tailed distribution within a high-dimensional, temporally sequential space. As a first step towards quantitative verification, we propose a computational method that leverages sequential adversarial attacks to identify weaknesses in learned locomotion controllers. Our research demonstrates that, even state-of-the-art robust controllers can fail significantly under well-designed, low-magnitude adversarial sequence. Through experiments in simulation and on the real robot, we validate our approach's effectiveness, and we illustrate how the results it generates can be used to robustify the original policy and offer valuable insights into the safety of these black-box policies. Project page: https://fanshi14.github.io/me/rss24.html

Figures (15)

• Figure 1: The state-of-the-art robust locomotion policy miki2022scirob can be destabilized even on flat ground when subjected to a sequence of low-magnitude multi-modal adversarial attacks on observations, demonstrating the vulnerabilities in learning-based neural controllers.
• Figure 2: The learned attack sequences under different methods, both able to cause the controller to fail in simulation. (a) is from the vanilla reinforcement learning method, whose adversarial output is in the bang-bang fashion and less practical; (b) is from the proposed method, whose adversarial output is smooth and more realistic for real-world scenarios. As shown in Fig. \ref{['figure:state-estimator-GT']}, real-world errors do not frequently oscillate between positive and negative values.
• Figure 3: Didactic example to show the proposed method being effective. (a)-(b): Stage 1: we limit the $y$ values of the adversarial forces to be positive (c)-(d): Stage 3: we allow the $y$ values of the adversarial forces to be either positive or negative when attacking the finetuned controller. During reattacking, the learned adversary effectively finds the unpatched weakness by leveraging the previously unused negative $y$ forces.
• Figure 4: Plots of adversarial output to fall over the real robot. (a) is the adversarial attack sequence in the command space; (b) is the attack sequence in the observation space, which outputs the orientation errors within $3^{\circ}$. Note that to avoid the damage on the physical hardware, early-stop is triggered before the robot falls down.
• Figure 5: Orientation data from the state estimator and ground truth (via motion capture) when the robot traverses flat indoor terrain, with peak errors reaching almost $3^{\circ}$. These estimation errors tend to significantly increase on uneven or slippery outdoor surfaces, or when the robot is subjected to perturbations bloesch2013state.