Towards Physical World Backdoor Attacks against Skeleton Action Recognition

TL;DR

The paper introduces Physical Skeleton Backdoor Attacks (PSBA), the first physical backdoor framework for Skeleton Action Recognition that embeds trigger actions through joint-angle manipulations using inverse/forward kinematics. It formalizes the attacker objective to maximize target-class activation when the trigger is present while preserving clean-data performance, and demonstrates both poison-label and clean-label variants, including a trigger-enhancing strategy to boost efficacy. Across three SAR models and three datasets, PSBA achieves high attack success rates and exhibits robustness against several defenses, with additional validation on a real-world Kinect V2 dataset. The work underscores important security implications for SAR systems and motivates the development of dedicated defenses against physically realizable backdoors.

Abstract

Skeleton Action Recognition (SAR) has attracted significant interest for its efficient representation of the human skeletal structure. Despite its advancements, recent studies have raised security concerns in SAR models, particularly their vulnerability to adversarial attacks. However, such strategies are limited to digital scenarios and ineffective in physical attacks, limiting their real-world applicability. To investigate the vulnerabilities of SAR in the physical world, we introduce the Physical Skeleton Backdoor Attacks (PSBA), the first exploration of physical backdoor attacks against SAR. Considering the practicalities of physical execution, we introduce a novel trigger implantation method that integrates infrequent and imperceivable actions as triggers into the original skeleton data. By incorporating a minimal amount of this manipulated data into the training set, PSBA enables the system misclassify any skeleton sequences into the target class when the trigger action is present. We examine the resilience of PSBA in both poisoned and clean-label scenarios, demonstrating its efficacy across a range of datasets, poisoning ratios, and model architectures. Additionally, we introduce a trigger-enhancing strategy to strengthen attack performance in the clean label setting. The robustness of PSBA is tested against three distinct backdoor defenses, and the stealthiness of PSBA is evaluated using two quantitative metrics. Furthermore, by employing a Kinect V2 camera, we compile a dataset of human actions from the real world to mimic physical attack situations, with our findings confirming the effectiveness of our proposed attacks. Our project website can be found at https://qichenzheng.github.io/psba-website.

Figures (6)

• Figure 1: Illustration of our backdoor attacks against SAR. (a) shows how we implant the trigger action. Taking the case where "Bending sideways" is a trigger action as an example, we maintain the original action "salute" while bending sideways. (b) After injecting poisoned data into the training dataset, we train a model on such a dataset. At inference time, when the attacker performs a "shoot with gun" while bending sideways, the backdoor will be activated and mislead the model towards the target class "salute".
• Figure 2: (a): Schematic diagram of rotation using quaternion. (b): Illustration of joints. Purple joints from root joint $r$ to key joint $k$ undergo inverse kinematics, while positions of all colored joints except the root joint $r$ are transformed to the target position in forward kinematics. (b) illustrates that when using "bending sideways" as the trigger action, joint 1 serves as the root joint, and joint 2 serves as the key joint.
• Figure 3: (a): Trigger implantation diagram. (b): Trigger-enhancing strategy of C-PSBA.
• Figure 4: Ablation study on the with/without trigger-enhancing strategy for C-PSBA.
• Figure 5: Resistance of P-PSBA to STRIP. Poisoning ratio is set to 2%.