A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation

TL;DR

This work proposes a novel frame-work featuring two pose augmentors: the weak and the strong augmentors, and leverages meta-optimization to simulate domain shifts in the optimization process of the pose estimator, thereby improving its generalization ability.

Abstract

3D human pose data collected in controlled laboratory settings present challenges for pose estimators that generalize across diverse scenarios. To address this, domain generalization is employed. Current methodologies in domain generalization for 3D human pose estimation typically utilize adversarial training to generate synthetic poses for training. Nonetheless, these approaches exhibit several limitations. First, the lack of prior information about the target domain complicates the application of suitable augmentation through a single pose augmentor, affecting generalization on target domains. Moreover, adversarial training's discriminator tends to enforce similarity between source and synthesized poses, impeding the exploration of out-of-source distributions. Furthermore, the pose estimator's optimization is not exposed to domain shifts, limiting its overall generalization ability. To address these limitations, we propose a novel framework featuring two pose augmentors: the weak and the strong augmentors. Our framework employs differential strategies for generation and discrimination processes, facilitating the preservation of knowledge related to source poses and the exploration of out-of-source distributions without prior information about target poses. Besides, we leverage meta-optimization to simulate domain shifts in the optimization process of the pose estimator, thereby improving its generalization ability. Our proposed approach significantly outperforms existing methods, as demonstrated through comprehensive experiments on various benchmark datasets.Our code will be released at \url{https://github.com/davidpengucf/DAF-DG}.

Figures (10)

• Figure 1: Comparisons between existing single-augmentor frameworks and our proposed dual-augmentor framework on a toy example. Current single-augmentor methods excel at simulating Target Domain 2 but exhibit limitations in simulating Target Domain 1, closely resembling the source, and Target Domain 3, deviating significantly from the source. In our framework, the weak augmentor excels in simulating Target Domain 1, while the strong augmentor effectively imitates both Target Domain 2 and 3.
• Figure 2: Overall framework of our dual-augmentor method. Initially, the original pose undergoes processing through two pose augmentors, resulting in weak- and strong-augmented poses (See Sec. \ref{['sec:aug']}). The weak augmentor simulates target domains similar to the source domain, while the strong augmentor emulates target domains that deviate significantly from the source distributions. Subsequently, the original pose and the two augmented poses are input to the pose estimator for further meta-optimization (See Sec. \ref{['sec:meta']}).
• Figure 3: The differentiation of the weak and strong generators. Within each pipeline, denoted as "W-" for weak ones and "S-" for strong ones, there exist four pose states: original (OR), after bone angle operation (BA), after bone length operation (BL) and after rotation and translation operation (RT). For proximate states, similarities are enhanced for both generators. When there is a one-state gap between states, the weak generator continues to enhance similarities, whereas the strong generator enlarges dissimilarities.
• Figure 4: Qualitative results on Cross-dataset evaluation. Left is 3DHP dataset, and right is 3DPW dataset.
• Figure 5: Results on Cross-scenario evaluation. Left is for task S1,S5,S6,S7,S8 $\rightarrow$ S9,S11, and right is for task S1,S5 $\rightarrow$ S6,S7,S8.