IP-Bench: Benchmark for Image Protection Methods in Image-to-Video Generation Scenarios

Abstract

With the rapid advancement of image-to-video (I2V) generation models, their potential for misuse in creating malicious content has become a significant concern. For instance, a single image can be exploited to generate a fake video, which can be used to attract attention and gain benefits. This phenomenon is referred to as an I2V generation misuse. Existing image protection methods suffer from the absence of a unified benchmark, leading to an incomplete evaluation framework. Furthermore, these methods have not been systematically assessed in I2V generation scenarios and against preprocessing attacks, which complicates the evaluation of their effectiveness in real-world deployment scenarios.To address this challenge, we propose IP-Bench (Image Protection Bench), the first systematic benchmark designed to evaluate protection methods in I2V generation scenarios. This benchmark examines 6 representative protection methods and 5 state-of-the-art I2V models. Furthermore, our work systematically evaluates protection methods' robustness with two robustness attack strategies under practical scenarios and analyzes their cross-model & cross-modality transferability. Overall, IP-Bench establishes a systematic, reproducible, and extensible evaluation framework for image protection methods in I2V generation scenarios.

Figures (4)

• Figure 1: IP-Bench benchmarks image protection methods in terms of protection effectiveness, visual fidelity, robustness against attacks, and cross-model & cross-modality transferability. The figure shows the overall pipeline of our benchmark.
• Figure 2: Protection effectiveness across I2V models.Average VBench Degradation Rate and FVD scores. For both metrics, higher values (warmer colors) indicate stronger protection. Detailed per-dimension scores are provided in \ref{['fig:protected_result_radar']}.
• Figure 3: The radar chart of VBench Scores in four different dimensions for 6 protection methods across 5 different I2V models.
• Figure 4: Heatmap visualization of methods' performances under preprocessing attacks. The first row shows the VBench metrics, and the second row shows the FVD metrics. The left column displays the results under the JPEG Compression attack, while the right column shows the results under the Gaussian Noise attack.