Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FrameShield: Adversarially Robust Video Anomaly Detection

Mojtaba Nafez, Mobina Poulaei, Nikan Vasei, Bardia Soltani Moakhar, Mohammad Sabokrou, MohammadHossein Rohban

TL;DR

The paper tackles the vulnerability of weakly supervised video anomaly detectors to adversarial perturbations by proposing FrameShield, a two-phase framework that first generates frame-level pseudo-labels with PromptMIL and then augments training with SRD-generated pseudo anomalies for precise frame-level supervision. The second stage performs frame-level adversarial training under a min-max objective with PGD, using an aggressive perturbation budget of $\epsilon = \frac{0.5}{255}$. Across five benchmarks, FrameShield achieves up to 71.0% improvements in robust AUROC under PGD-1000, with ablations confirming the value of both pseudo-labeling and SRD components. The work demonstrates that combining near-distribution pseudo anomalies with frame-level supervision yields substantially stronger defenses than prior MIL-based methods and offers a practical path toward robust WSVAD in real-world settings.

Abstract

Weakly Supervised Video Anomaly Detection (WSVAD) has achieved notable advancements, yet existing models remain vulnerable to adversarial attacks, limiting their reliability. Due to the inherent constraints of weak supervision, where only video-level labels are provided despite the need for frame-level predictions, traditional adversarial defense mechanisms, such as adversarial training, are not effective since video-level adversarial perturbations are typically weak and inadequate. To address this limitation, pseudo-labels generated directly from the model can enable frame-level adversarial training; however, these pseudo-labels are inherently noisy, significantly degrading performance. We therefore introduce a novel Pseudo-Anomaly Generation method called Spatiotemporal Region Distortion (SRD), which creates synthetic anomalies by applying severe augmentations to localized regions in normal videos while preserving temporal consistency. Integrating these precisely annotated synthetic anomalies with the noisy pseudo-labels substantially reduces label noise, enabling effective adversarial training. Extensive experiments demonstrate that our method significantly enhances the robustness of WSVAD models against adversarial attacks, outperforming state-of-the-art methods by an average of 71.0\% in overall AUROC performance across multiple benchmarks. The implementation and code are publicly available at https://github.com/rohban-lab/FrameShield.

FrameShield: Adversarially Robust Video Anomaly Detection

TL;DR

The paper tackles the vulnerability of weakly supervised video anomaly detectors to adversarial perturbations by proposing FrameShield, a two-phase framework that first generates frame-level pseudo-labels with PromptMIL and then augments training with SRD-generated pseudo anomalies for precise frame-level supervision. The second stage performs frame-level adversarial training under a min-max objective with PGD, using an aggressive perturbation budget of . Across five benchmarks, FrameShield achieves up to 71.0% improvements in robust AUROC under PGD-1000, with ablations confirming the value of both pseudo-labeling and SRD components. The work demonstrates that combining near-distribution pseudo anomalies with frame-level supervision yields substantially stronger defenses than prior MIL-based methods and offers a practical path toward robust WSVAD in real-world settings.

Abstract

Weakly Supervised Video Anomaly Detection (WSVAD) has achieved notable advancements, yet existing models remain vulnerable to adversarial attacks, limiting their reliability. Due to the inherent constraints of weak supervision, where only video-level labels are provided despite the need for frame-level predictions, traditional adversarial defense mechanisms, such as adversarial training, are not effective since video-level adversarial perturbations are typically weak and inadequate. To address this limitation, pseudo-labels generated directly from the model can enable frame-level adversarial training; however, these pseudo-labels are inherently noisy, significantly degrading performance. We therefore introduce a novel Pseudo-Anomaly Generation method called Spatiotemporal Region Distortion (SRD), which creates synthetic anomalies by applying severe augmentations to localized regions in normal videos while preserving temporal consistency. Integrating these precisely annotated synthetic anomalies with the noisy pseudo-labels substantially reduces label noise, enabling effective adversarial training. Extensive experiments demonstrate that our method significantly enhances the robustness of WSVAD models against adversarial attacks, outperforming state-of-the-art methods by an average of 71.0\% in overall AUROC performance across multiple benchmarks. The implementation and code are publicly available at https://github.com/rohban-lab/FrameShield.
Paper Structure (33 sections, 10 equations, 4 figures, 21 tables, 1 algorithm)

This paper contains 33 sections, 10 equations, 4 figures, 21 tables, 1 algorithm.

Table of Contents

  1. Related Work
  2. Preliminaries
  3. Methods
  4. First Phase: PromptMIL Training
  5. Second Phase: Adversarial Training
  6. Experiments
  7. Ablation Study
  8. Conclusion
  9. Dataset Details
  10. Performance Evaluation of the PromptMIL Framework
  11. Alternative Pseudo-Labeling Strategies to the PromptMIL Framework
  12. Analysis of Sensitivity to the Pseudo-Labeling Threshold $\tau$
  13. Detailed Analysis of $\epsilon$ Values During Training and Testing
  14. Effect of Label Noise in Adversarial Training
  15. Robustness Evaluation Against PGD Attacks with $\ell_2$ Norm
  16. ...and 18 more sections

Figures (4)

  • Figure 1: Overview of the FrameShield Framework: (1) The WSVAD training set is constructed using frame-level labels for normal data and frame-level pseudo labels for weakly labeled real anomaly data. Additionally, the Spatiotemporal Region Distortion (SRD) module generates pseudo-anomaly samples with precise frame-level annotations, further augmenting the training set with adversarial perturbations. (2) Two predefined prompts, "normal" and "anomaly," are used to extract text embeddings from the frozen text encoder. (3) Training videos are segmented into chunks and processed by the XClip-based encoder. The dot product between each chunk's feature representation and the text prompt embeddings is computed to obtain normality and abnormality scores, optimizing the network through chunk-level cross-entropy loss.
  • Figure 2: Visualization of Spatiotemporal Region Distortion (SRD): A synthetic anomaly generation method for precise frame-level labeling, applying harsh augmentations to Grad-CAM-identified foreground regions and simulating motion irregularities with random curved vectors across frames.
  • Figure 3: Visualization of pseudo-anomalous data generated by the SRD module. The top row shows the original (normal) sequence of frames along with the Grad-CAM heatmap of the first frame. The bottom row displays the corresponding distorted (pseudo-anomalous) frames created by applying the SRD distortion strategy.
  • Figure 4: Visualization of pseudo-anomalous data generated by the SRD module. The top row shows the original (normal) sequence of frames along with the Grad-CAM heatmap of the first frame. The bottom row displays the corresponding distorted (pseudo-anomalous) frames created by applying the SRD distortion strategy.