CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation

TL;DR

CycleNet addresses unpaired image-to-image translation with pre-trained diffusion models by introducing cycle-consistency regularization within a single denoising network conditioned on text and input images. Built on a ControlNet-augmented Stable Diffusion backbone, it adds forward-backward translation losses and a self-regularization term to enforce domain fidelity, while CLIP-based prompts enable flexible text guidance. The approach delivers competitive image quality and translation fidelity, demonstrates notable out-of-domain generalization with simple prompt changes, and remains practical under limited data and single-GPU training. These findings highlight a data-efficient, scalable direction for diffusion-based manipulation and potential extensions to coherent video prediction and physical-state editing.

Abstract

Diffusion models (DMs) have enabled breakthroughs in image synthesis tasks but lack an intuitive interface for consistent image-to-image (I2I) translation. Various methods have been explored to address this issue, including mask-based methods, attention-based methods, and image-conditioning. However, it remains a critical challenge to enable unpaired I2I translation with pre-trained DMs while maintaining satisfying consistency. This paper introduces Cyclenet, a novel but simple method that incorporates cycle consistency into DMs to regularize image manipulation. We validate Cyclenet on unpaired I2I tasks of different granularities. Besides the scene and object level translation, we additionally contribute a multi-domain I2I translation dataset to study the physical state changes of objects. Our empirical studies show that Cyclenet is superior in translation consistency and quality, and can generate high-quality images for out-of-domain distributions with a simple change of the textual prompt. Cyclenet is a practical framework, which is robust even with very limited training data (around 2k) and requires minimal computational resources (1 GPU) to train. Project homepage: https://cyclenetweb.github.io/

Figures (13)

• Figure 1: A high-resolution example of CycleNet for diffusion-based image-to-image translation compared to other diffusion-based methods. CycleNet produces high-quality translations with satisfactory consistency. The areas in the boxes are enlarged for detailed comparisons.
• Figure 2: The image translation cycle includes a forward translation from $x_0$ to $\Bar{y}_0$ and a backward translation to $\Bar{x}_0$. The key idea of our method is to ensure that when conditioned on an image $c_{\mathrm{img}}$ that falls into the target domain specified by $c_{\mathrm{text}}$, the LDM should reproduce this image condition through the reverse process. The dashed lines indicate the regularization in the loss functions.
• Figure 3: Step skipping during sampling. The source image is from MSCOCO lin2014microsoft.
• Figure 4: Qualitative comparison in scene/object-level tasks. CycleNet produces high-quality translations with satisfactory consistency. The areas in the boxes are enlarged for detailed comparisons.
• Figure 5: Qualitative comparison in coffee$\leftrightarrow$empty and empty$\leftrightarrow$juice tasks.

Theorems & Definitions (7)

• Proposition 1: Cycle Consistency Regularization
• Proposition 2: Self Regularization
• proof
• proof
• proof
• proof
• proof