Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models

TL;DR

Pose-guided image synthesis faces major challenges when source and target poses differ, limiting realism and pose fidelity.This work introduces Progressive Conditional Diffusion Models (PCDMs), a three-stage framework comprising a prior conditional diffusion model for global target features, an inpainting diffusion model to establish dense correspondence and generate a coarse image, and a refining diffusion model to restore texture and ensure fine-detail consistency.Extensive experiments on DeepFashion and Market-1501—along with user studies and a downstream re-identification task—demonstrate strong quantitative performance (SSIM, LPIPS, FID) and superior realism compared with state-of-the-art methods.While achieving high-quality results, the approach incurs higher computational cost due to the multi-stage design, pointing to future work on efficiency improvements and faster inference.

Abstract

Recent work has showcased the significant potential of diffusion models in pose-guided person image synthesis. However, owing to the inconsistency in pose between the source and target images, synthesizing an image with a distinct pose, relying exclusively on the source image and target pose information, remains a formidable challenge. This paper presents Progressive Conditional Diffusion Models (PCDMs) that incrementally bridge the gap between person images under the target and source poses through three stages. Specifically, in the first stage, we design a simple prior conditional diffusion model that predicts the global features of the target image by mining the global alignment relationship between pose coordinates and image appearance. Then, the second stage establishes a dense correspondence between the source and target images using the global features from the previous stage, and an inpainting conditional diffusion model is proposed to further align and enhance the contextual features, generating a coarse-grained person image. In the third stage, we propose a refining conditional diffusion model to utilize the coarsely generated image from the previous stage as a condition, achieving texture restoration and enhancing fine-detail consistency. The three-stage PCDMs work progressively to generate the final high-quality and high-fidelity synthesized image. Both qualitative and quantitative results demonstrate the consistency and photorealism of our proposed PCDMs under challenging scenarios.The code and model will be available at https://github.com/tencent-ailab/PCDMs.

Figures (19)

• Figure 1: Existing methods typically utilize unaligned image-to-image generation at the conditional level.
• Figure 2: The three-stage pipeline of Progressive Conditional Diffusion Models (PCDMs) progressively operates to generate the final high-quality and high-fidelity synthesized image. Our approach progressively predicts the global features, dense correspondences, and texture restoration of target image, enabling image synthesis.
• Figure 3: Illustration of the prior conditional diffusion model. The prior conditional diffusion model uses pose coordinates and global alignment relationship of the image to predict the global features of the target image.
• Figure 4: Overview of the inpainting conditional diffusion model. The inpainting conditional diffusion model utilizes the global features obtained from the previous stage to establish dense correspondences.
• Figure 5: Illustrative of the refining conditional diffusion model. The refining conditional diffusion model leverages the coarse-grained image generated in the previous stage to rectify textures and ensure consistency.