TraceMark-LDM: Authenticatable Watermarking for Latent Diffusion Models via Binary-Guided Rearrangement

TL;DR

TraceMark-LDM addresses the risk of unregulated AIGC by introducing a non-destructive, latent-space watermarking scheme for Latent Diffusion Models. It embeds a binary watermark through binary and group rearrangements of Gaussian latent samples, guided by the watermark, and strengthens robustness via encoder fine-tuning and a robust extraction procedure that uses DDIM inversion. The approach achieves superior attribution accuracy and resilience against image processing, VAE-based, and diffusion-based regeneration attacks while maintaining image quality comparable to state-of-the-art methods. This method enables reliable content tracing and model attribution in practical AIGC deployments, with potential extensions to video generation and structured diffusion pipelines.

Abstract

Image generation algorithms are increasingly integral to diverse aspects of human society, driven by their practical applications. However, insufficient oversight in artificial Intelligence generated content (AIGC) can facilitate the spread of malicious content and increase the risk of copyright infringement. Among the diverse range of image generation models, the Latent Diffusion Model (LDM) is currently the most widely used, dominating the majority of the Text-to-Image model market. Currently, most attribution methods for LDMs rely on directly embedding watermarks into the generated images or their intermediate noise, a practice that compromises both the quality and the robustness of the generated content. To address these limitations, we introduce TraceMark-LDM, an novel algorithm that integrates watermarking to attribute generated images while guaranteeing non-destructive performance. Unlike current methods, TraceMark-LDM leverages watermarks as guidance to rearrange random variables sampled from a Gaussian distribution. To mitigate potential deviations caused by inversion errors, the small absolute elements are grouped and rearranged. Additionally, we fine-tune the LDM encoder to enhance the robustness of the watermark. Experimental results show that images synthesized using TraceMark-LDM exhibit superior quality and attribution accuracy compared to state-of-the-art (SOTA) techniques. Notably, TraceMark-LDM demonstrates exceptional robustness against various common attack methods, consistently outperforming SOTA methods.

Figures (6)

• Figure 1: Application scenarios for TraceMark-LDM. The service provider encodes the regular user's information into a watermark and hides the watermark in the generated image. For different cases, watermarks can be designed either to identify whether a given image was generated by the service model (model detection) or to trace the user who generated a specific image (user attribution). When the image is tampered with or infringed upon by malicious users, the extracted watermark can facilitate model detection and user attribution, thereby safeguard the copyright of the regular user.
• Figure 2: Illustration of the denoising and inversion process and analysis of inversion errors. (a) Visualizes the denoising and inversion process in the latent diffusion model (LDM). The original image $I$ is encoded into the latent space, and the latent variable $z_0$ is recovered through inversion, while the attacked image $I_{\text{attack}}$ introduces further discrepancies in the recovered latent variables. (b) Presents the distribution of inversion errors under different conditions, showing the error values between $z_T$ and its reconstructed counterparts ($\bar{z}_T$, $z_T'$, and $z_T^*$). (c) Evaluates the sign consistency of latent elements, highlighting that elements with larger absolute values ($|z_T| \geq 0.675$) exhibit higher robustness to inversion errors compared to smaller absolute values ($|z_T| < 0.675$).
• Figure 3: The framework of TraceMark-LDM. The watermark information $m$ is used to guide a rearrangement process to produce a latent input $z_{wt}$ for generating watermarked image $I$. During the fine-tuning phase, the decoder parameters are frozen and only the encoder parameters are updated. The extraction process is an inversion of the embedding.
• Figure 4: Performance comparison of different watermarking methods under various attack types and intensities, evaluated using bit accuracy as the performance metric. The plots illustrate the robustness of methods against (a) median filtering, (b) JPEG compression, (c) Gaussian blur, (d) Gaussian noise, (e) salt-and-pepper noise, (f) resizing and restoring, (g) Regen-VAE-A, and (h) Regen-VAE-B.
• Figure 5: Detection accuracy of watermarking methods under diffusion-based regeneration attacks.