Bidirectional Regression for Monocular 6DoF Head Pose Estimation and Reference System Alignment

TL;DR

This work tackles monocular 6DoF head pose estimation by addressing the entanglement between facial geometry and depth, and by fixing cross-dataset evaluation biases arising from inconsistent head-center definitions. It introduces TRGv2, a lightweight iterative framework that jointly regresses 6DoF pose and dense 3D facial landmarks through a landmark-to-image projection loop, enforcing metric consistency and generalizing translation through correction parameters within a pinhole camera model. A key contribution is the reference-system alignment strategy, which quantifies and corrects translation bias between datasets to enable fair cross-dataset comparisons. Across ARKitFace, BIWI, and DD-Pose, TRGv2 achieves state-of-the-art accuracy and efficiency, including notable improvements in ADD and translation errors, and its design supports real-time deployment on modest GPU hardware. The work also provides new annotated landmarks for DD-Pose and releases code, offering a practical pathway for robust, cross-domain 6DoF head pose estimation in real-world scenarios.

Abstract

Precise six-degree-of-freedom (6DoF) head pose estimation is crucial for safety-critical applications and human-computer interaction scenarios, yet existing monocular methods still struggle with robust pose estimation. We revisit this problem by introducing TRGv2, a lightweight extension of our previous Translation, Rotation, and Geometry (TRG) network, which explicitly models the bidirectional interaction between facial geometry and head pose. TRGv2 jointly infers facial landmarks and 6DoF pose through an iterative refinement loop with landmark-to-image projection, ensuring metric consistency among face size, rotation, and depth. To further improve generalization to out-of-distribution data, TRGv2 regresses correction parameters instead of directly predicting translation, combining them with a pinhole camera model for analytic depth estimation. In addition, we identify a previously overlooked source of bias in cross-dataset evaluations due to inconsistent head center definitions across different datasets. To address this, we propose a reference system alignment strategy that quantifies and corrects translation bias, enabling fair comparisons across datasets. Extensive experiments on ARKitFace, BIWI, and the challenging DD-Pose benchmarks demonstrate that TRGv2 outperforms state-of-the-art methods in both accuracy and efficiency. Code and newly annotated landmarks for DD-Pose will be publicly available.

Figures (11)

• Figure 1: Illustration of different methods for inferring 6DoF head pose. Optimization-based methods zielonka2022micaguo2022jmlr2023_tip_perspnet first predict the face geometry and then calculate the head pose sequentially. The regression-based approach albiero2021img2pose directly regresses the head pose from the input image. In contrast, the proposed method jointly estimates both the face geometry and the head pose to fully exploit the synergy between them.
• Figure 2: Overall pipeline of TRGv2. TRGv2 is designed with a focus on the bidirectional interaction between face geometry and 6DoF head pose. It first performs initial predictions of face geometry and head pose, and then refines these estimates using a structure that enforces metric consistency between the two through a landmark-to-image projection strategy.
• Figure 3: Illustration of the calculation of head translation $T_t$ from the correction parameters $c_t$ and the bounding box information $I_\text{bbox}$. Best viewed in color.
• Figure 4: Left: Visualization of the BIWI face mesh (blue) and the ARKitFace face mesh (gold), aligned at the same head center. Right: Predicted 3D dense landmarks and head pose from a model trained on the ARKitFace training data. The BIWI landmarks denote the 3D dense landmarks provided by the BIWI dataset. The white numbers in the second column indicate the translation error ($\text{MAE}_t$). The red and blue dots represent the ground-truth and predicted head centers, respectively.
• Figure 5: Distributions of ground-truth translation and correction parameters in the ARKitFace and BIWI datasets. The colors blue, green, and brown represent the ARKitFace training data, ARKitFace test data, and the BIWI dataset, respectively. The symbol $*$ denotes the ground-truth.