Cross-Dataset Gaze Estimation by Evidential Inter-intra Fusion

TL;DR

This work tackles cross-dataset gaze estimation by addressing distribution shifts across datasets. It introduces Evidential Inter-intra Fusion (EIF), a framework that maintains independent single-dataset branches to prevent source-domain degradation while adding a cross-dataset fusion branch to harness shared information, using $G$ local regressors per gaze subspace and an overlap parameter $\\alpha$. The model employs evidential regressors based on the Normal-Inverse-Gamma distribution and fuses them through a Mixture of Normal-Inverse-Gamma (MoNIG) to produce gaze predictions with quantified uncertainty. Two-stage training enables rapid adaptation to new dataset combinations, and experiments across ETH-XGaze, Gaze360, MPIIGaze, and EyeDiap show consistent improvements in both source and unseen domains, with meaningful domain adaptation gains when limited target data are available. The approach yields actionable uncertainty estimates and provides a scalable pathway to robust gaze estimation in diverse real-world settings.

Abstract

Achieving accurate and reliable gaze predictions in complex and diverse environments remains challenging. Fortunately, it is straightforward to access diverse gaze datasets in real-world applications. We discover that training these datasets jointly can significantly improve the generalization of gaze estimation, which is overlooked in previous works. However, due to the inherent distribution shift across different datasets, simply mixing multiple dataset decreases the performance in the original domain despite gaining better generalization abilities. To address the problem of ``cross-dataset gaze estimation'', we propose a novel Evidential Inter-intra Fusion EIF framework, for training a cross-dataset model that performs well across all source and unseen domains. Specifically, we build independent single-dataset branches for various datasets where the data space is partitioned into overlapping subspaces within each dataset for local regression, and further create a cross-dataset branch to integrate the generalizable features from single-dataset branches. Furthermore, evidential regressors based on the Normal and Inverse-Gamma (NIG) distribution are designed to additionally provide uncertainty estimation apart from predicting gaze. Building upon this foundation, our proposed framework achieves both intra-evidential fusion among multiple local regressors within each dataset and inter-evidential fusion among multiple branches by Mixture \textbfof Normal Inverse-Gamma (MoNIG distribution. Experiments demonstrate that our method consistently achieves notable improvements in both source domains and unseen domains.

Figures (9)

• Figure 1: Comparison between traditional Cross-Domain scenario and our Cross-Dataset Training scenario. Here, $\mathcal{D}_s$ represents the source domain, $\mathcal{D}_t$ represents the target domain. Cross-Dataset Training involves training on multiple source domains with significant distribution discrepancies, aiming for consistent performance across source domains and target domains.
• Figure 2: Test accuracy on different datasets with varied combination of training data.
• Figure 3: Illustration of the proposed Evidential Inter-intra Fusion (EIF) framework. The model consists of multiple single-dataset branches and a cross-dataset branch, with each branch comprising multiple local regressors. We utilize the intra-evidential fusion module to perform local regressor fusion within each branch and the inter-evidential fusion module to perform fusion across multiple branches. These evidence fusion modules are formulated without any learnable parameters. The output includes both predictions and uncertainty. Due to gaze estimation involving yaw and pitch components, there are two identical regression layers with the same model structure.
• Figure 4: Parameter Sensitivity Analysis.
• Figure 5: Visualizing Weights During Inter-Intra Fusion.