Unsupervised Synthetic Image Attribution: Alignment and Disentanglement

TL;DR

This work tackles the problem of attributing synthetic images to their training data without access to paired supervision. It introduces Alignment and Disentanglement (A&D), a two-stage unsupervised framework that first uses contrastive self-supervised learning to align concepts across synthetic and exemplar domains, then applies Infomax ICA to disentangle representations while preserving cross-domain alignment, with an identity-initialized, orthogonal constraint to prevent permutation. The authors provide a theoretical connection to Canonical Correlation Analysis, showing that their two-stage process approximates CCA without paired data. Experiments on the AbC benchmark show that A&D can match or exceed supervised approaches across multiple backbones, illustrating the practicality and robustness of unsupervised synthetic image attribution. The work offers a fresh perspective on cross-domain concept matching and lays groundwork for copyright protection and model transparency in open and closed-model settings alike.

Abstract

As the quality of synthetic images improves, identifying the underlying concepts of model-generated images is becoming increasingly crucial for copyright protection and ensuring model transparency. Existing methods achieve this attribution goal by training models using annotated pairs of synthetic images and their original training sources. However, obtaining such paired supervision is challenging, as it requires either well-designed synthetic concepts or precise annotations from millions of training sources. To eliminate the need for costly paired annotations, in this paper, we explore the possibility of unsupervised synthetic image attribution. We propose a simple yet effective unsupervised method called Alignment and Disentanglement. Specifically, we begin by performing basic concept alignment using contrastive self-supervised learning. Next, we enhance the model's attribution ability by promoting representation disentanglement with the Infomax loss. This approach is motivated by an interesting observation: contrastive self-supervised models, such as MoCo and DINO, inherently exhibit the ability to perform simple cross-domain alignment. By formulating this observation as a theoretical assumption on cross-covariance, we provide a theoretical explanation of how alignment and disentanglement can approximate the concept-matching process through a decomposition of the canonical correlation analysis objective. On the real-world benchmarks, AbC, we show that our unsupervised method surprisingly outperforms the supervised methods. As a starting point, we expect our intuitive insights and experimental findings to provide a fresh perspective on this challenging task.

Figures (12)

• Figure 1: Synthetic image attribution without model access. Modern generative models are often closed-source and trained on vast datasets, making it challenging to attribute a generated image to its true training exemplar(s) when the model is inaccessible.
• Figure 2: Framework of Alignment and Disentanglement. We begin by extracting roughly aligned features using a shared pretrained backbone ($\mathbf{\Phi}$) through self-supervised contrastive learning. These features are then refined to be more independent and identifiable using two distinct linear mappings, $H_S$ and $H_E$, optimized with Infomax loss (ICA). To support cross-domain alignment and prevent harmful permutations introduced by ICA, we apply identity initialization and orthogonal constraints. This framework effectively functions as a CCA method without paired annotations.
• Figure 3: Data generation process for CCA. A shared latent variable $Z$ generates two observation views, $X_S$ and $X_E$, through different transformations.
• Figure 4: Examples of Exemplar (top) and Synthetic (bottom) images in the AbC benchmark.
• Figure 5: Attribution results visualization. We start with a synthetic image generated by CustomDiffusion (based on Stable Diffusion) and retrieve its exemplars from a pool of 1 million LAION images, showing the top 10 results. Alignment denotes attribution using the pretrained DINO model, Supervised refers to DINO fine-tuned with paired training data from the AbC benchmark, and A&D is our proposed unsupervised fine-tuning method. Red bounding boxes indicate the ground-truth exemplars.