Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EOPose : Exemplar-based object reposing using Generalized Pose Correspondences

Sarthak Mehrotra, Rishabh Jain, Mayur Hemani, Balaji Krishnamurthy, Mausoom Sarkar

TL;DR

This work tackles exemplar-based object reposing for generic objects by transferring pose from a target exemplar image while preserving texture and geometry. It introduces EOPose, an end-to-end GAN-based pipeline that first detects generalized keypoint correspondences, then performs a coarse pose-guided warping, and finally conducts fine-grained re-rendering with texture injection to recover occluded details. The approach is trained on a newly created Objaverse-derived paired dataset and is validated through quantitative metrics (SSIM, LPIPS, FID), qualitative comparisons, and a user study showing strong preference for EOPose over baselines. The results demonstrate that the combination of dense flow-based warping with texture-aware re-rendering reduces hallucinations and maintains brand-specific details, making it valuable for e-commerce and other applications requiring reliable variant generation.

Abstract

Reposing objects in images has a myriad of applications, especially for e-commerce where several variants of product images need to be produced quickly. In this work, we leverage the recent advances in unsupervised keypoint correspondence detection between different object images of the same class to propose an end-to-end framework for generic object reposing. Our method, EOPose, takes a target pose-guidance image as input and uses its keypoint correspondence with the source object image to warp and re-render the latter into the target pose using a novel three-step approach. Unlike generative approaches, our method also preserves the fine-grained details of the object such as its exact colors, textures, and brand marks. We also prepare a new dataset of paired objects based on the Objaverse dataset to train and test our network. EOPose produces high-quality reposing output as evidenced by different image quality metrics (PSNR, SSIM and FID). Besides a description of the method and the dataset, the paper also includes detailed ablation and user studies to indicate the efficacy of the proposed method

EOPose : Exemplar-based object reposing using Generalized Pose Correspondences

TL;DR

This work tackles exemplar-based object reposing for generic objects by transferring pose from a target exemplar image while preserving texture and geometry. It introduces EOPose, an end-to-end GAN-based pipeline that first detects generalized keypoint correspondences, then performs a coarse pose-guided warping, and finally conducts fine-grained re-rendering with texture injection to recover occluded details. The approach is trained on a newly created Objaverse-derived paired dataset and is validated through quantitative metrics (SSIM, LPIPS, FID), qualitative comparisons, and a user study showing strong preference for EOPose over baselines. The results demonstrate that the combination of dense flow-based warping with texture-aware re-rendering reduces hallucinations and maintains brand-specific details, making it valuable for e-commerce and other applications requiring reliable variant generation.

Abstract

Reposing objects in images has a myriad of applications, especially for e-commerce where several variants of product images need to be produced quickly. In this work, we leverage the recent advances in unsupervised keypoint correspondence detection between different object images of the same class to propose an end-to-end framework for generic object reposing. Our method, EOPose, takes a target pose-guidance image as input and uses its keypoint correspondence with the source object image to warp and re-render the latter into the target pose using a novel three-step approach. Unlike generative approaches, our method also preserves the fine-grained details of the object such as its exact colors, textures, and brand marks. We also prepare a new dataset of paired objects based on the Objaverse dataset to train and test our network. EOPose produces high-quality reposing output as evidenced by different image quality metrics (PSNR, SSIM and FID). Besides a description of the method and the dataset, the paper also includes detailed ablation and user studies to indicate the efficacy of the proposed method
Paper Structure (33 sections, 7 equations, 8 figures, 2 tables)

This paper contains 33 sections, 7 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Virtual Try-On
  4. Mask-Guided Object Composition Methods
  5. 3D Pose Representation
  6. Methodology
  7. Keypoint Correspondence Detection
  8. Coarse-Alignment with Pose-guided Warping
  9. Image Warping
  10. Losses
  11. Fine-grained Re-rendering
  12. Pose Encoder
  13. Texture Encoder
  14. Texture Injection Block
  15. tRGB Block
  16. ...and 18 more sections

Figures (8)

  • Figure 1: Exemplar-based object reposing involves synthesizing an object in a desired pose denoted by another image of a similar kind of object
  • Figure 2: A) Keypoint Detection: This module uses a pre-trained DINO-ViT to detect visual correspondences between the given pose image $I_p$ and appearance image $I_a$ B) Warping Module: This module takes in concatenated $I_a P_a, I_p, P_p$ as input and predicts a 2D flow used to deform the appearance image $I_a$ to align with the required pose denoted by pose image $I_p$
  • Figure 3: Our Generator module consumes the warped image $I_{wrp}$ to generate the final reposed output. It utilizes a series of texture injection blocks to inject multi-scale appearance features into the pose encoding for the image generation process
  • Figure 4: Paired dataset preparation by rendering Objaverseobjaverse 3d models of different objects in different poses for EOPose network training.
  • Figure 5: In this figure we show improvements along different qualitative aspects compared to thin plate spline and ControlCom zhang2023controlcom. We emphasize the differences in preserving pose (a,g,h), maintaining geometric integrity (b,e), and texture integrity (c,f,g,h).
  • ...and 3 more figures