VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking

TL;DR

VideoShield tackles the challenge of content control for diffusion-based video generation by introducing training-free in-generation watermarking that embeds a watermark during video synthesis. It maps watermark bits $m$ to template bits $TP$ and Gaussian noise via $Z^{T}$, enabling both watermark extraction through DDIM Inversion and tamper localization across temporal and spatial dimensions, with a Hierarchical Spatial-Temporal Refinement (HSTR) and Partial Threshold Binarization (PTB) to boost accuracy. Key contributions include the introduction of template bits, a reversible m ↔ TP ↔ Z^{T} ↔ FR chain, and the first temporal tamper localization module, plus a PTB/HSTR framework that enhances spatial localization without training. Experiments across multiple T2V/I2V models show robust watermark extraction and tamper localization while maintaining video quality, and the approach generalizes to image generation as well. The method offers practical impact for protecting the integrity and provenance of AI-generated video content in real-world deployments.

Abstract

Artificial Intelligence Generated Content (AIGC) has advanced significantly, particularly with the development of video generation models such as text-to-video (T2V) models and image-to-video (I2V) models. However, like other AIGC types, video generation requires robust content control. A common approach is to embed watermarks, but most research has focused on images, with limited attention given to videos. Traditional methods, which embed watermarks frame-by-frame in a post-processing manner, often degrade video quality. In this paper, we propose VideoShield, a novel watermarking framework specifically designed for popular diffusion-based video generation models. Unlike post-processing methods, VideoShield embeds watermarks directly during video generation, eliminating the need for additional training. To ensure video integrity, we introduce a tamper localization feature that can detect changes both temporally (across frames) and spatially (within individual frames). Our method maps watermark bits to template bits, which are then used to generate watermarked noise during the denoising process. Using DDIM Inversion, we can reverse the video to its original watermarked noise, enabling straightforward watermark extraction. Additionally, template bits allow precise detection for potential temporal and spatial modification. Extensive experiments across various video models (both T2V and I2V models) demonstrate that our method effectively extracts watermarks and detects tamper without compromising video quality. Furthermore, we show that this approach is applicable to image generation models, enabling tamper detection in generated images as well. Codes and models are available at https://github.com/hurunyi/VideoShield.

Figures (24)

• Figure 1: The overall framework of VideoShield. (1) In the Watermark Embedding and Extraction stage, we first map the watermark bits $m$ to the initial Gaussian noise $Z^{T}$ via an intermediate set of random template bits $TP$, which are derived from $m$. The video diffusion model $\mathcal{M}$ then iteratively denoises $Z^{T}$, ultimately generating the video frames $FR$. For watermark extraction, $\mathcal{M}$ uses DDIM Inversion on the tampered or distorted video frames $\overline{FR}$ to recover the inverted noise $\overline{Z^{T}}$. This noise is then transformed into inverted bits $IV$, from which the watermark bits are extracted. (2) In the Tamper Localization stage, $TP$ and $IV$ are processed by a temporal module to localize temporal tamper and restore their temporal positions. The resulting comparison bits matrix $CMP$ is then passed to a spatial module, which incorporates a hierarchical spatio-temporal refinement (HSTR) module to enhance localization performance. Both stages are training-free and can be applied to any diffusion-based video generation model.
• Figure 2: The pipeline of the temporal tamper localization module. We use the first frame ($IV_1$) inside the module (cyan area) as an example to show the localization process.
• Figure 3: The pipeline of the spatial tamper localization module containing a Hierarchical Spatial-Temporal Refinement (HSTR) module. Here, we show the internal workflow of HSTR when the total hierarchical level $L=3$.
• Figure 4: Comparison accuracy distributions of different local values on the watermarked and original videos generated by ModelScope and Stable-Video-Diffusion. Each data point in the figure represents the average accuracy of the sub-region with different local values $\mu$ ($\mu = 2^{l-1}$), containing different numbers of comparison bits.
• Figure 5: Some visual examples comparison of different spatial localization methods.