FMVP: Masked Flow Matching for Adversarial Video Purification

TL;DR

FMVP tackles adversarial vulnerabilities in video recognition by combining a masking-based disruption of adversarial structure with Conditional Flow Matching (CFM) for inpainting-like reconstruction. A Frequency-Gated Loss $\mathcal{L}_{\mathrm{FGL}}$ suppresses high-frequency adversarial noise while preserving low-frequency semantic content, and two training paradigms (Attack-Aware and Generalist) handle known and unknown threats. Empirical results on UCF-101 and HMDB-51 show FMVP outperforms state-of-the-art defenses under PGD, CW, and adaptive DiffHammer attacks, achieving robust accuracy above $87\%$ (PGD) and above $89\%$ (CW), with strong zero-shot adversarial detection (AUC up to $0.98$ for PGD). The method also offers fast inference via an Euler-based purification path, and its velocity-based signals enable cross-model transferability and practical deployment in secure video recognition tasks.

Abstract

Video recognition models remain vulnerable to adversarial attacks, while existing diffusion-based purification methods suffer from inefficient sampling and curved trajectories. Directly regressing clean videos from adversarial inputs often fails to recover faithful content due to the subtle nature of perturbations; this necessitates physically shattering the adversarial structure. Therefore, we propose Flow Matching for Adversarial Video Purification FMVP. FMVP physically shatters global adversarial structures via a masking strategy and reconstructs clean video dynamics using Conditional Flow Matching (CFM) with an inpainting objective. To further decouple semantic content from adversarial noise, we design a Frequency-Gated Loss (FGL) that explicitly suppresses high-frequency adversarial residuals while preserving low-frequency fidelity. We design Attack-Aware and Generalist training paradigms to handle known and unknown threats, respectively. Extensive experiments on UCF-101 and HMDB-51 demonstrate that FMVP outperforms state-of-the-art methods (DiffPure, Defense Patterns (DP), Temporal Shuffling (TS) and FlowPure), achieving robust accuracy exceeding 87% against PGD and 89% against CW attacks. Furthermore, FMVP demonstrates superior robustness against adaptive attacks (DiffHammer) and functions as a zero-shot adversarial detector, attaining detection accuracies of 98% for PGD and 79% for highly imperceptible CW attacks.

Figures (15)

• Figure 1: During inference, unlike DiffPure (a) which shifts the distribution via noise injection (purple) and FlowPure (b) which initiates directly from the original adversarial distribution (red), ours (c) employs masking to physically disrupt adversarial patterns. This shifts the input to a specific masked adversarial distribution (blue), effectively shattering the attack structure while preserving the original semantics via inpainting-based reconstruction.
• Figure 2: Overview of FMVP. Adversarial videos are transformed from the Adversarial Space $\mathcal{A}$ to the Masked Adversarial Space $\mathcal{M}$. This process disrupts adversarial patterns while preserving the original semantics of adjacent pixels. The training phase is conducted under the constraints of the Mean Squared Error (MSE) loss and the Frequency-Gated Loss within the Conditional Flow Matching (CFM) framework, while inference employs the Euler iterative update.
• Figure 3: The trajectories of 50 real samples in the space from clean $\rightarrow$ adversarial $\rightarrow$ masked $\rightarrow$ purified. Left: CW attack; right: PGD attack.
• Figure 4: Visualization of the Frequency-Gated Loss properties. (Left) The 2D spectral weight mask $\mathbf{W}$ shows that high weights concentrate in the low-frequency center. (Right) The 1D decay profile demonstrates the exponential drop in importance as frequency increases.
• Figure 5: Distribution of detection scores and ROC curves. Adversarial samples usually score higher than clean ones (horizontal axis), especially for PGD, while CW’s imperceptibility causes partial score overlap with clean samples.