SSI-DM: Singularity Skipping Inversion of Diffusion Models

TL;DR

The paper tackles the problem of inverting real images into diffusion-model noise space, identifying a fundamental singularity of the score function near $t=0$ that makes inversion ill-posed and yields non-Gaussian inverted noise. It introduces Singularity Skipping Inversion of Diffusion Models (SSI-DM), which bypasses the ill-conditioned region by injecting a small amount of Gaussian noise at a skipping time $t_{\text{SSI}}$ and then performing standard reverse-time integration, resulting in Gaussian-like inverted noise with high editability and fidelity. The approach is plug-and-play across DDIM, EDM, and Stable Diffusion, and is supported by theoretical analysis of the score singularity and a reconstruction–editability tradeoff bound, plus extensive experiments on LSUN Bedroom-256 and ImageNet-256 showing improved inversion quality, reconstruction metrics, and interpolation quality. The method extends naturally to both Variance Exploding and Variance Preserving processes, offering a practical, efficient solution for diffusion-model inversion with broad applicability to editing tasks and beyond.

Abstract

Inverting real images into the noise space is essential for editing tasks using diffusion models, yet existing methods produce non-Gaussian noise with poor editability due to the inaccuracy in early noising steps. We identify the root cause: a mathematical singularity that renders inversion fundamentally ill-posed. We propose Singularity Skipping Inversion of Diffusion Models (SSI-DM), which bypasses this singular region by adding small noise before standard inversion. This simple approach produces inverted noise with natural Gaussian properties while maintaining reconstruction fidelity. As a plug-and-play technique compatible with general diffusion models, our method achieves superior performance on public image datasets for reconstruction and interpolation tasks, providing a principled and efficient solution to diffusion model inversion.

Figures (15)

• Figure 1: Interpolations of real images with diverse styles and kinds based on Stable Diffusion using SSI-DM.
• Figure 2: Inverted noise contains image-dependent patterns. We visualize a single channel from the inverted noise as grayscale images for two pixel-space models, showing clear correlation with the original images.
• Figure 3: SSI method overview: Starting from a clean image $\mathbf{x}_0$ and adding noise to this image, we sample $\mathbf{\hat{x}}_{t_\text{SSI}}\sim p(\mathbf{x};\sigma_{t_\text{SSI}})$ at an early time $t_\text{SSI}$ to bypass the singularity near $t=0$, then iteratively solve the reverse ODE through an alternative trajectory to obtain $\mathbf{\hat{x}}_T$. Different noise samples allow distinct paths that faithfully reconstruct $\mathbf{x}_0$. By avoiding ill-conditioned early-time steps, SSI genrates high quality and editable Gaussian noises and reduces computational costs compared to methods requiring extra corrections.
• Figure 4: Visualization of $\|\mathbb{E}[\mathbf{x}_0|\mathbf{x}_t]-\mathbf{x}_t\|/\sigma_t$ along sampling trajectories. For distributions with analytically computable score functions (Fig. \ref{['fig:eight_point']}), the true score function exhibits $1/\sigma_t$ divergence as $t \to 0$, while neural network approximations show significant relative errors in this regime. Pretrained image diffusion models on CIFAR-10 (Fig. \ref{['fig:edm']}) demonstrate the same characteristic behavior.
• Figure 5: Cosine similarity between different inverted noise vectors (SSI-DM) and reconstruction results.

Theorems & Definitions (7)

• Proposition 4.1
• proof
• Theorem 3: Singularity of score function in VE SDE
• Remark 4
• Theorem 5: Concentration of the projected distance
• proof
• Remark 6