StyleFool: Fooling Video Classification Systems via Style Transfer

TL;DR

This work tackles the security of video classification by introducing StyleFool, a black-box unrestricted adversarial attack that uses style transfer to nudge non-semantic visual content toward classifier decision boundaries. The method combines color-proximity-based style selection, temporally coherent style transfer, and gradient-free optimization (NES and PGD) to produce effective, perceptually natural adversarial videos with far fewer queries than prior attacks. It demonstrates strong attack performance on UCF-101 and HMDB-51 against C3D and I3D, often achieving 100% attack success with substantially reduced query counts, while human users struggle to distinguish adversarial videos from clean ones. The paper also analyzes resilience against several defenses (AdvIT, ComDefend, RS) and discusses limitations of current defenses against unrestricted, style-transfer-based perturbations, highlighting the need for new defense strategies that account for non-norm-bounded perturbations.

Abstract

Video classification systems are vulnerable to adversarial attacks, which can create severe security problems in video verification. Current black-box attacks need a large number of queries to succeed, resulting in high computational overhead in the process of attack. On the other hand, attacks with restricted perturbations are ineffective against defenses such as denoising or adversarial training. In this paper, we focus on unrestricted perturbations and propose StyleFool, a black-box video adversarial attack via style transfer to fool the video classification system. StyleFool first utilizes color theme proximity to select the best style image, which helps avoid unnatural details in the stylized videos. Meanwhile, the target class confidence is additionally considered in targeted attacks to influence the output distribution of the classifier by moving the stylized video closer to or even across the decision boundary. A gradient-free method is then employed to further optimize the adversarial perturbations. We carry out extensive experiments to evaluate StyleFool on two standard datasets, UCF-101 and HMDB-51. The experimental results demonstrate that StyleFool outperforms the state-of-the-art adversarial attacks in terms of both the number of queries and the robustness against existing defenses. Moreover, 50% of the stylized videos in untargeted attacks do not need any query since they can already fool the video classification model. Furthermore, we evaluate the indistinguishability through a user study to show that the adversarial samples of StyleFool look imperceptible to human eyes, despite unrestricted perturbations.

Figures (9)

• Figure 1: An overview of StyleFool against video classification systems.
• Figure 2: Examples of stylized images: (a) the original image; (b) style transfer results using artistic styles; (c) style transfer result of our approach and the selected style. The introduction of color theme proximity produces more indistinguishable style transfer, compared with the use of artistic styles.
• Figure 3: Results of the user study on indistinguishability: (a) Participants' responses in the naturalness test. : very unnatural, : very natural; (b) Participants' responses in the realness test.
• Figure 4: Confidence scores of the original class and target class before and after style transfer with different style selection strategies.
• Figure 5: Visualization of StyleFool. The left four screenshots are taken from original videos, the right four screenshots are taken from adversarial videos, the image in the middle is the selected style image.