GRPose: Learning Graph Relations for Human Image Generation with Pose Priors

TL;DR

GRPose tackles pose-guided human image generation by modeling pose parts as a graph and propagating information through a Progressive Graph Integrator. It introduces a Graph Pose Adapter and a Pose Perception Loss based on a pretrained pose estimator to enforce pose fidelity. On the Human-Art and LAION-Human datasets, GRPose achieves superior pose alignment (higher AP and SAP, lower PCE) while maintaining competitive image quality compared to state-of-the-art methods. The approach is plug-and-play with existing diffusion backbones, enabling robust multi-pose generation and practical impact for animation and content creation.

Abstract

Recent methods using diffusion models have made significant progress in human image generation with various control signals such as pose priors. However, existing efforts are still struggling to generate high-quality images with consistent pose alignment, resulting in unsatisfactory output. In this paper, we propose a framework that delves into the graph relations of pose priors to provide control information for human image generation. The main idea is to establish a graph topological structure between the pose priors and latent representation of diffusion models to capture the intrinsic associations between different pose parts. A Progressive Graph Integrator (PGI) is designed to learn the spatial relationships of the pose priors with the graph structure, adopting a hierarchical strategy within an Adapter to gradually propagate information across different pose parts. Besides, a pose perception loss is introduced based on a pretrained pose estimation network to minimize the pose differences. Extensive qualitative and quantitative experiments conducted on the Human-Art and LAION-Human datasets clearly demonstrate that our model can achieve significant performance improvement over the latest benchmark models. The code is available at \url{https://xiangchenyin.github.io/GRPose/}.

Figures (7)

• Figure 1: Examples of the pose-guided human image generation task. The first row illustrates the generated results of the ControlNet and our GRPose methods while the second row visualizes the pose alignment across different base models. GRPose generates better results by well aligning with pose prior and scaling outputs to $512 \times 512$ pixels.
• Figure 2: Overview of Graph Relation Pose (GRPose). The Pose Encoder is adopted to capture multi-level scales of pose priors within a hierarchical structure, where a Progressive Graph Integrator is incorporated to capture graph relationships between different pose parts. The Pose Perception Loss adopts a pre-trained pose estimation network to regularize the pose alignment.
• Figure 3: Details of Progressive Graph Integrator (PGI). The pose prior $x_p$ and latent representation $x_l$ are gridded to construct graphs $\mathcal{G}_p$ and $\mathcal{G}_l$ respectively, where GCNs are employed to fuse and update the information.
• Figure 4: Two cases of Multi-Pose Generation. Our model outperforms ControlNet in generating multiple poses.
• Figure 5: Qualitative Results of our GRPose with different base diffusion models. We compared SD1.5, Anime Art and Realistic models of different styles.