Uncertainty-oriented Order Learning for Facial Beauty Prediction

TL;DR

This work addresses generalization gaps in facial beauty prediction caused by inconsistent evaluation standards across datasets and variability in human judgments. It introduces Uncertainty-oriented Order Learning (UOL), which models ratings as $z \sim \mathcal{N}(\mu(x), \Sigma(x))$ on a psychological scale and uses a distribution-comparison module with Monte Carlo sampling to learn robust order relations. A Wasserstein-distance-based hinge loss and a Bradley-Terry score estimator translate learned orders into FB scores, enabling reliable scoring even with unbalanced reference sets. Across SCUT-FBP5500 and several related datasets, UOL delivers improved accuracy and stronger cross-dataset generalization, demonstrating robustness to both standard bias and cognitive uncertainty in facial beauty evaluation.

Abstract

Previous Facial Beauty Prediction (FBP) methods generally model FB feature of an image as a point on the latent space, and learn a mapping from the point to a precise score. Although existing regression methods perform well on a single dataset, they are inclined to be sensitive to test data and have weak generalization ability. We think they underestimate two inconsistencies existing in the FBP problem: 1. inconsistency of FB standards among multiple datasets, and 2. inconsistency of human cognition on FB of an image. To address these issues, we propose a new Uncertainty-oriented Order Learning (UOL), where the order learning addresses the inconsistency of FB standards by learning the FB order relations among face images rather than a mapping, and the uncertainty modeling represents the inconsistency in human cognition. The key contribution of UOL is a designed distribution comparison module, which enables conventional order learning to learn the order of uncertain data. Extensive experiments on five datasets show that UOL outperforms the state-of-the-art methods on both accuracy and generalization ability.

Figures (5)

• Figure 1: Two inconsistencies in FBP problem. (a). Three images, coming from SCUT-FBP5500, Hot-Or-Not and MEBeauty datasets respectively, have similar normalized FB scores but different FB appearances. (b). Ratings of a face image from different people are commonly inconsistent. Many FBP methods take the mean of these ratings as the FB score.
• Figure 2: (a) The training phase of UOL. The order of distributions is constrained by cross entropy loss and hinge loss, and the dispersion of the distributions is constrained by KL loss. (b) The estimation phase of UOL. In uncertainty modeling, the FB of a facial image is modeled by a multi-dimensional Gaussian distribution whose mean $\mu$ and diagonal covariance $\Sigma$ are learned by VGG from the image. In distribution comparison, we sample from both the distributions of test image and reference image to form a pair and predict its order by a comparator in order learning. After having the order relations of $T$ pairs between reference images and the test image, the Bradley-Terry model is applied to estimate the score of the test image.
• Figure 3: In our UOL, FB features of facial images are modeled as the multi-dimensional Gaussian distributions on psychological scale space instead of fixed points on the conventional latent space.
• Figure 4: Monte Carlo sampling of our distribution comparison module.
• Figure 5: Illustration of Bradley-Terry (BT) estimation module. (a) shows the probability distribution of BT model. (b) shows the data distribution of SCUT-FBP5500 dataset. (c) lists the reference set, all reference images with precise scores are going to be compared with the input image to estimate its FB score. The range marked by blue box is the reference set selected by MC rules ref29, which cannot cover the entire range.