Enhancing accuracy of uncertainty estimation in appearance-based gaze tracking with probabilistic evaluation and calibration

TL;DR

This paper tackles the challenge of unreliable uncertainty estimation in appearance-based gaze tracking due to training-data biases. It introduces a strictly proper scoring metric based on coverage probability and a post-hoc probabilistic calibration workflow using an isotonic regression regressor to align predicted and observed distributions, applied independently to pitch and yaw. The approach is validated on MPIIGaze and RTGene with ResNet backbones, showing consistent reductions in coverage probability error (CPE) and improvements in angular prediction, as well as more reliable 95% confidence intervals in case studies. The work contributes a principled evaluation framework and a practical calibration method that enhances trustworthiness for downstream applications like driver monitoring and human–computer interaction.

Abstract

Accurately knowing uncertainties in appearance-based gaze tracking is critical for ensuring reliable downstream applications. Due to the lack of individual uncertainty labels, current uncertainty-aware approaches adopt probabilistic models to acquire uncertainties by following distributions in the training dataset. Without regulations, this approach lets the uncertainty model build biases and overfits the training data, leading to poor performance when deployed. We first presented a strict proper evaluation metric from the probabilistic perspective based on comparing the coverage probability between prediction and observation to provide quantitative evaluation for better assessment on the inferred uncertainties. We then proposed a correction strategy based on probability calibration to mitigate biases in the estimated uncertainties of the trained models. Finally, we demonstrated the effectiveness of the correction strategy with experiments performed on two popular gaze estimation datasets with distinctive image characteristics caused by data collection settings.

Figures (7)

• Figure 1: Inaccuracies of existing uncertainty-aware gaze tracking model and the proposed remedy plan. Uncertainty estimations from existing models provide inaccurate prediction ranges and often cannot capture the ground truth gaze angle values (black dot) because of strong biases towards the training data from the uncertainty model. We proposed to introduce a calibration model to compensate for the bias behavior to generate accurate uncertainty estimates such that majority of the ground truth value of gaze angles can be captured for better downstream decision making.
• Figure 2: Demonstration of uncertainty inaccuracy causes. The predicted cumulative probability distribution is different from the actual cumulative probability distribution from observation, deviating from the idea case (red dashed line). When model predicted cumulative probability distribution to be 0.9 the real probability is less than 0.82, leading to overconfident estimation or underestimation of uncertainties.
• Figure 3: Visualization of MPIIGaze and RTGene dataset samples. Samples in RTGene is much noisier than the MPIIGaze image. Difference in the two dataset provides sharp probability distribution to test the effectiveness of calibration.
• Figure 4: The uncertainty-aware appearance-based gaze estimation structure used in the experiments. The model takes inputs of left and right eye patches together with head angles to output two normal distributions to describe the gaze angle estimates. Mean values are angular estimates, and variances represent uncertainty.
• Figure 5: Visualization of correction effect of the calibration process. The calibration model learns the error behavior from a small subset (black dots) collected from testing data to learn model’s error behavior. During testing, the model can apply appropriate corrections to the uncalibrated uncertainty model such that the output from the correct model (cyan dots) matches the ideal case closer to reduce uncertainty error, rather than the high-error uncalibrated uncertainty inferences (blue dots).