TruePose: Human-Parsing-guided Attention Diffusion for Full-ID Preserving Pose Transfer

TL;DR

TruePose addresses the clothing pattern preservation challenge in diffusion-based pose-guided person image synthesis by incorporating a human-parsing-guided attention mechanism. The method uses a Siamese network with dual UNets (SourceNet and TargetNet), augmented by a human-parsing-guided fusion attention (HPFA) and CLIP-guided attention alignment (CAA) to embed source face and garment details into target generation. Empirical results on DeepFashion and WPose show significant improvements over 13 baselines in preserving facial identity and clothing patterns, with ablations confirming the complementary benefits of HPFA and CAA. The approach enables high-quality, full-ID-preserving pose transfers and supports practical fashion applications such as style transfer and region-based editing without retraining.

Abstract

Pose-Guided Person Image Synthesis (PGPIS) generates images that maintain a subject's identity from a source image while adopting a specified target pose (e.g., skeleton). While diffusion-based PGPIS methods effectively preserve facial features during pose transformation, they often struggle to accurately maintain clothing details from the source image throughout the diffusion process. This limitation becomes particularly problematic when there is a substantial difference between the source and target poses, significantly impacting PGPIS applications in the fashion industry where clothing style preservation is crucial for copyright protection. Our analysis reveals that this limitation primarily stems from the conditional diffusion model's attention modules failing to adequately capture and preserve clothing patterns. To address this limitation, we propose human-parsing-guided attention diffusion, a novel approach that effectively preserves both facial and clothing appearance while generating high-quality results. We propose a human-parsing-aware Siamese network that consists of three key components: dual identical UNets (TargetNet for diffusion denoising and SourceNet for source image embedding extraction), a human-parsing-guided fusion attention (HPFA), and a CLIP-guided attention alignment (CAA). The HPFA and CAA modules can embed the face and clothes patterns into the target image generation adaptively and effectively. Extensive experiments on both the in-shop clothes retrieval benchmark and the latest in-the-wild human editing dataset demonstrate our method's significant advantages over 13 baseline approaches for preserving both facial and clothes appearance in the source image.

Figures (7)

• Figure 1: Pose-guided person image synthesis (PGPIS) task and comparison among CFLD lu2024coarse, PCDM (ICLR'24) shen2024advancing, and our methods. The two SOTA methods fail to preserve the clothing patterns and textures. The main reason is that the image encoder overlooks the clothing details (See the "Feature Attention Map (Feat. Att. Map)"). In contrast, our method can generate high-quality images with preserved face and clothing patterns.
• Figure 2: Top: comparing average attention scores of different regions within source images. Bottom: visualization results of attention maps of our method and two baseline methods.
• Figure 3: Pipeline of the proposed human-parsing-guided attention diffusion model.
• Figure 4: Qualitative comparisons with PISE zhang2021pise, ADGANmen2020controllable, SPGNet lv2021learning, CASD zhou2022cross, DPTN zhang2022exploring, PIDM bhunia2023person, NTED ren2022neural, PCDM shen2024advancing and CFLD lu2024coarse.
• Figure 5: User study results in terms of R2G,G2R and Jab metrics. Higher values in metrics means better quality of generated results.