PCDiff: Proactive Control for Ownership Protection in Diffusion Models with Watermark Compatibility

TL;DR

PCDiff tackles diffusion-model IP protection by replacing passive watermarking with proactive access control, embedding a FuserKey-driven fuser and fine-tuning layers inside the decoder to gate high-quality generation. The approach freezes the base model and trains only the added layers, coupling cryptographic validation with the diffusion process and preserving watermark traceability. Experiments on Stable Diffusion v2.1 show that authorized users get high-quality outputs while unauthorized use yields degraded imagery, and watermark signatures remain verifiable under attack; the key space of $2^{128}$ and layer-complexity ensure strong security with moderate overhead. Overall, PCDiff demonstrates a practical and scalable pathway for proactive ownership enforcement in diffusion models while maintaining compatibility with existing watermarking techniques.

Abstract

With the growing demand for protecting the intellectual property (IP) of text-to-image diffusion models, we propose PCDiff -- a proactive access control framework that redefines model authorization by regulating generation quality. At its core, PCDIFF integrates a trainable fuser module and hierarchical authentication layers into the decoder architecture, ensuring that only users with valid encrypted credentials can generate high-fidelity images. In the absence of valid keys, the system deliberately degrades output quality, effectively preventing unauthorized exploitation.Importantly, while the primary mechanism enforces active access control through architectural intervention, its decoupled design retains compatibility with existing watermarking techniques. This satisfies the need of model owners to actively control model ownership while preserving the traceability capabilities provided by traditional watermarking approaches.Extensive experimental evaluations confirm a strong dependency between credential verification and image quality across various attack scenarios. Moreover, when combined with typical post-processing operations, PCDIFF demonstrates powerful performance alongside conventional watermarking methods. This work shifts the paradigm from passive detection to proactive enforcement of authorization, laying the groundwork for IP management of diffusion models.

Figures (6)

• Figure 1: An illustration of the motivation behind PCDiff.
• Figure 2: The training strategy and framework of proposed ownership protection method for diffusion models.
• Figure 3: Visual comparison of images generated under different structures and conditions
• Figure 4: Relationship between image quality and theoretical cracking time for various PCDiff configurations, as measured by PSNR, SSIM, and FID—with estimated cracking times plotted on a base‑10 logarithmic scale. Note that the SSIM values are multiplied by 10 in the figure.
• Figure 5: Visual comparison: First row shows images from a randomly selected mid block (a), a randomly selected upsampling layer (b), and both (c). Second row shows images with some mid blocks removed (d), some upsampling layers removed (e), and both removed (f).