Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation

PengFei Zheng, Yonggang Zhang, Zhen Fang, Tongliang Liu, Defu Lian, Bo Han

TL;DR

NoiseDiffusion performs interpolation within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss and enables us to interpolate natural images without causing artifacts or information loss, thus achieving the best interpolation results.

Abstract

Image interpolation based on diffusion models is promising in creating fresh and interesting images. Advanced interpolation methods mainly focus on spherical linear interpolation, where images are encoded into the noise space and then interpolated for denoising to images. However, existing methods face challenges in effectively interpolating natural images (not generated by diffusion models), thereby restricting their practical applicability. Our experimental investigations reveal that these challenges stem from the invalidity of the encoding noise, which may no longer obey the expected noise distribution, e.g., a normal distribution. To address these challenges, we propose a novel approach to correct noise for image interpolation, NoiseDiffusion. Specifically, NoiseDiffusion approaches the invalid noise to the expected distribution by introducing subtle Gaussian noise and introduces a constraint to suppress noise with extreme values. In this context, promoting noise validity contributes to mitigating image artifacts, but the constraint and introduced exogenous noise typically lead to a reduction in signal-to-noise ratio, i.e., loss of original image information. Hence, NoiseDiffusion performs interpolation within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss. Consequently, NoiseDiffusion enables us to interpolate natural images without causing artifacts or information loss, thus achieving the best interpolation results.

NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation

TL;DR

NoiseDiffusion performs interpolation within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss and enables us to interpolate natural images without causing artifacts or information loss, thus achieving the best interpolation results.

Abstract

Image interpolation based on diffusion models is promising in creating fresh and interesting images. Advanced interpolation methods mainly focus on spherical linear interpolation, where images are encoded into the noise space and then interpolated for denoising to images. However, existing methods face challenges in effectively interpolating natural images (not generated by diffusion models), thereby restricting their practical applicability. Our experimental investigations reveal that these challenges stem from the invalidity of the encoding noise, which may no longer obey the expected noise distribution, e.g., a normal distribution. To address these challenges, we propose a novel approach to correct noise for image interpolation, NoiseDiffusion. Specifically, NoiseDiffusion approaches the invalid noise to the expected distribution by introducing subtle Gaussian noise and introduces a constraint to suppress noise with extreme values. In this context, promoting noise validity contributes to mitigating image artifacts, but the constraint and introduced exogenous noise typically lead to a reduction in signal-to-noise ratio, i.e., loss of original image information. Hence, NoiseDiffusion performs interpolation within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss. Consequently, NoiseDiffusion enables us to interpolate natural images without causing artifacts or information loss, thus achieving the best interpolation results.
Paper Structure (31 sections, 7 theorems, 42 equations, 30 figures)

This paper contains 31 sections, 7 theorems, 42 equations, 30 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. The details of diffusion models
  5. Image editing
  6. The Image Interpolation Methods
  7. The spherical linear interpolation of images
  8. The reason for failure
  9. Introducing noise for interpolation
  10. NoiseDiffusion
  11. Boundary control
  12. The connection of methods
  13. Experiments
  14. The lubricating coefficient
  15. The change in style
  16. ...and 16 more sections

Key Result

Theorem 1

The standard normal distribution $\mathcal{N}(\mathbf{0},\bm{I}_n)$ in high dimensions is close to the uniform distribution on the sphere of radius $\sqrt{n}$.

Figures (30)

  • Figure 1: Comparison of images generated with different interpolation methods.
  • Figure 2: The spherical linear interpolation. Original images: The images on the left are natural images, whereas the images on the right are generated by the diffusion model. Interpolation results: The images on the left and right are the interpolation results of natural images and images generated by diffusion model respectively.
  • Figure 3: The impact of noise levels. We added Gaussian noise with levels of $\sigma(t) = [70, 75, 80, 85, 90]$ to the image on the left. Subsequently, we applied denoising to each noisy image with the same noise level of $\sigma(t') = 80$, resulting in the denoised images on the right.
  • Figure 4: Introducing noise for image interpolation. In the interpolated images, the top one represents the interpolation result with less Gaussian noise, while the bottom one represents the interpolation result with more Gaussian noise.
  • Figure 5: The impact of lubricating coefficient $\gamma$.
  • ...and 25 more figures

Theorems & Definitions (13)

  • Theorem 1
  • Theorem 2
  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Definition 1
  • Lemma 2
  • proof
  • Lemma 3
  • ...and 3 more