PTDiffusion: Free Lunch for Generating Optical Illusion Hidden Pictures with Phase-Transferred Diffusion Model

TL;DR

PTDiffusion addresses the challenge of generating optical illusion hidden pictures by translating a reference image into a text-guided scene while preserving recognizable structural cues. It introduces a training-free, plug-and-play Phase-Transferred Diffusion framework that performs phase-based fusion of reference structure with semantic prompts in the latent diffusion feature space, via inversion, reconstruction, and sampling trajectories. A phase transfer module transfers guidance-phase information through 2D FFT-based mixing, with a decayed blending schedule, and an asynchronous variant allows controllable hidden-content discernibility. The approach achieves state-of-the-art quality on illusion-picture synthesis without model training, improves text fidelity and visual naturalness, and provides flexible control over hidden-content visibility, demonstrated on diverse real and synthetic references. The method holds practical potential for artistic design, education, and content creation, and can be extended to other T2I backbones.

Abstract

Optical illusion hidden picture is an interesting visual perceptual phenomenon where an image is cleverly integrated into another picture in a way that is not immediately obvious to the viewer. Established on the off-the-shelf text-to-image (T2I) diffusion model, we propose a novel training-free text-guided image-to-image (I2I) translation framework dubbed as \textbf{P}hase-\textbf{T}ransferred \textbf{Diffusion} Model (PTDiffusion) for hidden art syntheses. PTDiffusion harmoniously embeds an input reference image into arbitrary scenes described by the text prompts, producing illusion images exhibiting hidden visual cues of the reference image. At the heart of our method is a plug-and-play phase transfer mechanism that dynamically and progressively transplants diffusion features' phase spectrum from the denoising process to reconstruct the reference image into the one to sample the generated illusion image, realizing deep fusion of the reference structural information and the textual semantic information in the diffusion model latent space. Furthermore, we propose asynchronous phase transfer to enable flexible control to the degree of hidden content discernability. Our method bypasses any model training and fine-tuning process, all while substantially outperforming related text-guided I2I methods in image generation quality, text fidelity, visual discernibility, and contextual naturalness for illusion picture synthesis, as demonstrated by extensive qualitative and quantitative experiments. Our project is publically available at \href{https://xianggao1102.github.io/PTDiffusion_webpage/}{this web page}.

Figures (27)

• Figure 1: Taking the first image on the left as an example, what do you see at your first glance? A painting of a path through a forest (zoom in for a detailed look), or a human face (zoom out for a more global view)? Based on the off-the-shelf text-to-image diffusion model, we contribute a plug-and-play method that naturally dissolves a reference image (shown in the bottom-right corner) into arbitrary scenes described by a text prompt, providing a free lunch for synthesizing optical illusion hidden pictures using diffusion model. Better zoom in.
• Figure 2: Overview of PTDiffusion. Built upon the pre-trained Latent Diffusion Model (LDM), PTDiffusion is composed of three diffusion trajectories. The inversion trajectory inverts the reference image into the LDM Gaussian noise space. The reconstruction trajectory recovers the reference image from the inverted noise embedding. The sampling trajectory samples the final illusion image from random noise guided by the text prompt. The reconstruction and sampling trajectory are bridged by our proposed phase transfer module, which dynamically transplants diffusion features' phase spectrum to smoothly blend source image structure with textual semantics in the LDM feature space.
• Figure 3: Illustration of the phase transfer module (PTM). To transfer the phase of $\hat{z}_{t}$ into $\tilde{z}_{t}$, we apply 2D FFT to decompose their magnitude $\hat{M}_{t}$, $\tilde{M}_{t}$ and phase $\hat{P}_{t}$, $\tilde{P}_{t}$, linearly fuse their phase with a blending coefficient $b_{t}$, and recombine the fused phase with $\tilde{M}_{t}$. Finally, the manipulated FFT feature is converted back to the spatial domain via 2D IFFT to form the phase-transferred $\tilde{z}_{t}$.
• Figure 4: Visualization of the guidance features {$\hat{z}_{t}$} along the 100-step reconstruction trajectory. The structural information of $\hat{z}_{t}$ becomes increasingly distinct as the denoising proceeds.
• Figure 5: Illustration of the asynchronous phase transfer which transfers phase across diffusion features at different time steps.