How to Trace Latent Generative Model Generated Images without Artificial Watermark?

TL;DR

The paper addresses tracing whether an image was produced by a specific latent diffusion model without adding artificial watermarks. It introduces LatentTracer, an alteration-free origin-attribution method that combines encoder-based latent initialization with gradient-based inversion to reconstruct images from the inspected model’s decoder. Across Stable Diffusion variants and Kandinsky 2.1, LatentTracer achieves high accuracy (roughly 93–98%) and significantly improves efficiency over prior reverse-engineering approaches, while preserving image quality. The findings suggest that current decoders impart an implicit watermark on decoded latent samples, with broad implications for model provenance, IP protection, and responsible AI deployment.

Abstract

Latent generative models (e.g., Stable Diffusion) have become more and more popular, but concerns have arisen regarding potential misuse related to images generated by these models. It is, therefore, necessary to analyze the origin of images by inferring if a particular image was generated by a specific latent generative model. Most existing methods (e.g., image watermark and model fingerprinting) require extra steps during training or generation. These requirements restrict their usage on the generated images without such extra operations, and the extra required operations might compromise the quality of the generated images. In this work, we ask whether it is possible to effectively and efficiently trace the images generated by a specific latent generative model without the aforementioned requirements. To study this problem, we design a latent inversion based method called LatentTracer to trace the generated images of the inspected model by checking if the examined images can be well-reconstructed with an inverted latent input. We leverage gradient based latent inversion and identify a encoder-based initialization critical to the success of our approach. Our experiments on the state-of-the-art latent generative models, such as Stable Diffusion, show that our method can distinguish the images generated by the inspected model and other images with a high accuracy and efficiency. Our findings suggest the intriguing possibility that today's latent generative generated images are naturally watermarked by the decoder used in the source models. Code: https://github.com/ZhentingWang/LatentTracer.

Figures (6)

• Figure 1: Pipeline of our latent inversion method. First, our method uses the corresponding encoder to get the starting point for the inversion. Then, it uses the gradient-based optimization to search the inverted latent by minimizing the reconstruction loss. The examined image is flagged as a belonging image of the inspected model if the final reconstruction loss is smaller than a threshold.
• Figure 2: Comparison on the reconstruction loss distributions for different inversion methods. The scenario is distinguishing the 500 images generated by the inspected model (i.e., Stable Diffusion v2-base) and the 500 images generated by other model (i.e., Stable Diffusion v1-5 here). 50 prompts sampled from PromptHero prompthero are used to generate these belonging images and non-belonging images (More details about the used prompts can be found in \ref{['sec:detailed_prompts']}). Our method is highly effective since the reconstruction losses for the belongings and that for non-belongings are nearly completely separated in our method.
• Figure 3: Effects of the initial distance to the ground-truth latent. The starting point closer to the ground-truth latent will lead to better efficiency and effectiveness for the inversion.
• Figure 4: Comparison on the reconstruction loss curve for the random starting point and the encoder-generated starting point. The encoder-based starting point lead to a faster convergence speed and a better separation of the reconstruction losses.
• Figure 5: Given a specific latent generative model, can we trace the images generated by this model without extra artificial watermarks? We show that the images generated by the inspected model can be traced without any additional requirements on the model's training and generation phase (such as adding a watermark after generation tancik2020stegastampwen2023tree or injecting fingerprinting during training yu2021artificialyu2022responsiblefernandez2023stable) by using the reconstruction loss computed by our method. For example, here we show some images generated by different models and their reconstruction losses on different models. Even though the generated images from different models can look seemingly identical, their reconstruction loss can differ by an order of magnitude. The reconstruction loss is extremely low if the examined image is generated by the inspected model.

Theorems & Definitions (5)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5