CLERF: Contrastive LEaRning for Full Range Head Pose Estimation

TL;DR

This work tackles the sparsity challenge in full-range head pose estimation by introducing CLERF, a contrastive learning framework that leverages anchor-positive synthetic pairs generated by a 3D-aware GAN and geometry-based augmentations to cover the entire pose space, including upside-down orientations. By preserving geodesic distances under rotation and flipping, CLERF constructs robust triplets and optimizes with Circle Loss, followed by a downstream MLP to predict rotation matrices via Gram–Schmidt normalization. The approach achieves state-of-the-art or competitive performance on standard FR-HPE benchmarks and exhibits clear robustness under minor rotations and flips, significantly outperforming existing full-range models on heavily transformed data. The combination of synthetic anchor-positive generation, SO(3)-aware augmentations, and contrastive representation learning provides a practical pathway to true full-range HPE with strong generalization to real-world variants.

Abstract

We introduce a novel framework for representation learning in head pose estimation (HPE). Previously such a scheme was difficult due to head pose data sparsity, making triplet sampling infeasible. Recent progress in 3D generative adversarial networks (3D-aware GAN) has opened the door for easily sampling triplets (anchor, positive, negative). We perform contrastive learning on extensively augmented data including geometric transformations and demonstrate that contrastive learning allows networks to learn genuine features that contribute to accurate HPE. On the other hand, we observe that existing HPE works struggle to predict head poses as accurately when test image rotation matrices are slightly out of the training dataset distribution. Experiments show that our methodology performs on par with state-of-the-art models on standard test datasets and outperforms them when images are slightly rotated/ flipped or full range head pose. To the best of our knowledge, we are the first to deliver a true full range HPE model capable of accurately predicting any head pose including upside-down pose. Furthermore, we compared with other existing full-yaw range models and demonstrated superior results.

Figures (6)

• Figure 1: Proposed method for contrastive learning in full range head pose representation, with a batch size of 3 instances for illustration purposes. Steps 1$\sim$6 comprise a training iteration for the representation model which is frozen in the downstream MLP training in step 7.
• Figure 2: Visualization of 300W-LP dataset zhu2017face after randomized rotation and flipping augmentations. A point on the sphere $\in \mathbb{R}^3$ is formed by projecting a rotation matrix to the 3D sphere, which multiplies unit vectors in (a) x, (b) y, and (c) z coordinate axes. The top row shows random rotation matrices (in green) along with those in the 300W-LP dataset (in blue), and the bottom row shows the augmented 300W-LP dataset (in red). The geometric transformations expand the original dataset for wider coverage.
• Figure 3: Change of model performance across different choices in (a) batch size, (b) epochs, and (c) backbone model. A batch size of 32, 16 epochs, and Swin Transformer make the best combination. (AFLW2000 is abbreviated as AFLW.)
• Figure 4: Head pose test predictions of (d) CLERF and (a)$\sim$(c) three baseline models versus the (e) ground truth on the original image (first row) and its SA (second row) and FA (third row) versions. Head pose is represented by the three lines colored in red, blue, and green. We observe barely noticeable differences between the model predictions in the original and SA test image, but CLERF much more accurately predicted the FA head pose.
• Figure 5: 3D TSNE plot of the embedding vectors of a video across (a)$\sim$(e) different models. The video shows a person turning around for a total of 360 degrees. The similarity of colors indicates their temporal proximity.

Theorems & Definitions (3)

• Theorem 3.1
• Corollary 3.1.1
• proof