WMVLM: Evaluating Diffusion Model Image Watermarking via Vision-Language Models

TL;DR

WMVLM addresses the need for a unified, interpretable evaluation of diffusion-model watermarking across both residual and semantic watermarks. It defines tailored quality and security metrics, linking residual artifacts to a PSNR-based score $v^{\text{(res)}}$ and leveraging latent-space distribution tests against $\mathcal{N}(0,I)$ to derive $p$-values for semantic watermarks with scores $v^{\text{(sem)}}, s^{\text{(sem)}}$. The framework trains a vision-language model through a three-stage pipeline—Category/Score Pre-training, Interpretability Cold Start via teacher distillation, and Generalization via Group Relative Policy Optimization (GRPO)—to output scores and human-readable explanations. Empirical results on Stable Diffusion show WMVLM outperforming state-of-the-art VLMs and generalizing across datasets, diffusion models, and watermarking methods, while delivering interpretable outputs alongside numerical metrics. Overall, WMVLM provides a practical, scalable tool for rigorous watermark evaluation and guidance for the design of diffusion-model watermarking techniques, with potential extensions to additional modalities.

Abstract

Digital watermarking is essential for securing generated images from diffusion models. Accurate watermark evaluation is critical for algorithm development, yet existing methods have significant limitations: they lack a unified framework for both residual and semantic watermarks, provide results without interpretability, neglect comprehensive security considerations, and often use inappropriate metrics for semantic watermarks. To address these gaps, we propose WMVLM, the first unified and interpretable evaluation framework for diffusion model image watermarking via vision-language models (VLMs). We redefine quality and security metrics for each watermark type: residual watermarks are evaluated by artifact strength and erasure resistance, while semantic watermarks are assessed through latent distribution shifts. Moreover, we introduce a three-stage training strategy to progressively enable the model to achieve classification, scoring, and interpretable text generation. Experiments show WMVLM outperforms state-of-the-art VLMs with strong generalization across datasets, diffusion models, and watermarking methods.

Figures (4)

• Figure 1: Comparison of evaluation paradigms. While traditional residual and semantic metrics provide only absolute numerical values, WMarkGPT introduces VLM-based interpretability for visible watermarks. Our WMVLM establishes a unified and interpretable framework specifically tailored to evaluate both visual quality and security in diffusion model image watermarking.
• Figure 2: WMVLM follows a three-stage training process. After defining the quality and security scores, the model undergoes Category and Score Pre-training, Interpretability Cold Start, and Generalization Enhancement via GRPO.
• Figure 3: Examples of residual watermarks evaluation.
• Figure 4: Examples of semantic watermarks evaluation.