Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Domain Image Conversion by CycleDM

Sho Shimotsumagari, Shumpei Takezaki, Daichi Haraguchi, Seiichi Uchida

TL;DR

This paper proposes a novel unpaired image-to-image domain conversion method, CycleDM, which incorporates the concept of CycleGAN into the diffusion model, and finds that its performs better than other comparable approaches.

Abstract

The purpose of this paper is to enable the conversion between machine-printed character images (i.e., font images) and handwritten character images through machine learning. For this purpose, we propose a novel unpaired image-to-image domain conversion method, CycleDM, which incorporates the concept of CycleGAN into the diffusion model. Specifically, CycleDM has two internal conversion models that bridge the denoising processes of two image domains. These conversion models are efficiently trained without explicit correspondence between the domains. By applying machine-printed and handwritten character images to the two modalities, CycleDM realizes the conversion between them. Our experiments for evaluating the converted images quantitatively and qualitatively found that ours performs better than other comparable approaches.

Cross-Domain Image Conversion by CycleDM

TL;DR

This paper proposes a novel unpaired image-to-image domain conversion method, CycleDM, which incorporates the concept of CycleGAN into the diffusion model, and finds that its performs better than other comparable approaches.

Abstract

The purpose of this paper is to enable the conversion between machine-printed character images (i.e., font images) and handwritten character images through machine learning. For this purpose, we propose a novel unpaired image-to-image domain conversion method, CycleDM, which incorporates the concept of CycleGAN into the diffusion model. Specifically, CycleDM has two internal conversion models that bridge the denoising processes of two image domains. These conversion models are efficiently trained without explicit correspondence between the domains. By applying machine-printed and handwritten character images to the two modalities, CycleDM realizes the conversion between them. Our experiments for evaluating the converted images quantitatively and qualitatively found that ours performs better than other comparable approaches.
Paper Structure (29 sections, 7 equations, 6 figures, 3 tables)

This paper contains 29 sections, 7 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Brief review of diffusion model
  4. Diffusion models for image conversion
  5. Diffusion models for character image generation
  6. CycleDM
  7. Overview
  8. Training CycleDM
  9. Cycle Consistency Loss
  10. Adversarial Loss
  11. Identity Loss
  12. Training the conversion models $F_t$ and $G_t$
  13. Experimental Results
  14. Dataset
  15. Implementation details
  16. ...and 14 more sections

Figures (6)

  • Figure 1: Cross-domain conversion task between machine-printed and handwritten character images. The converted image should resemble the original to some degree.
  • Figure 2: (a) SDEdit meng2022sdedit for cross-domain conversion. Here, a handwritten character image is converted to its machine-printed version, but it is straightforward to realize the conversion in the reverse direction. (b) and (c) Overview of the proposed CycleDM in its training phase and conversion phase, respectively.For simpler notations, $F_t$ and $G_t$ are used instead of $F_t$ and $G_t$.
  • Figure 3: Loss functions to train the conversion models $F_t$ and $G_t$ of CycleDM. For simplicity, $F_t$ and $G_t$ are denoted as $F$ and $G$ and the class condition $c$ is omitted. The backbone DDPM is pretrained, and its parameters are frozen during training $F_t$ and $G_t$.
  • Figure 4: Image conversion from the handwritten character domain to the machine-printed character domain (HW$\to$MP). The green boxes are attached to the results subjectively appropriate, whereas the red boxes are inappropriate.
  • Figure 5: Image conversion from the machine-printed character domain to the handwritten character domain (MP$\to$HW).
  • ...and 1 more figures