A Generalized Label Shift Perspective for Cross-Domain Gaze Estimation

TL;DR

The paper addresses cross-domain gaze estimation by reframing it as a Generalized Label Shift problem, involving shifts in both the label distribution and the conditional data-generating process. It introduces a GLS correction framework that jointly handles label reweighting and probability-aware conditional alignment, with a practical realization using a truncated Gaussian label model and PCOD-based alignment. The proposed GLSGE method demonstrates state-of-the-art performance across multiple backbones and datasets, and shows strong plug-in applicability to other gaze-estimation models, supported by ablations and visualization analyses. This approach provides a principled path to improve cross-domain generalization in gaze tasks, with potential extensions to other regression-like computer-vision problems and tasks with continuous labels. Limitations include the need for some target-domain data during training and the computational cost of kernel-based components, suggesting future work on prior-informed priors and scalable alternatives.

Abstract

Aiming to generalize the well-trained gaze estimation model to new target domains, Cross-domain Gaze Estimation (CDGE) is developed for real-world application scenarios. Existing CDGE methods typically extract the domain-invariant features to mitigate domain shift in feature space, which is proved insufficient by Generalized Label Shift (GLS) theory. In this paper, we introduce a novel GLS perspective to CDGE and modelize the cross-domain problem by label and conditional shift problem. A GLS correction framework is presented and a feasible realization is proposed, in which a importance reweighting strategy based on truncated Gaussian distribution is introduced to overcome the continuity challenges in label shift correction. To embed the reweighted source distribution to conditional invariant learning, we further derive a probability-aware estimation of conditional operator discrepancy. Extensive experiments on standard CDGE tasks with different backbone models validate the superior generalization capability across domain and applicability on various models of proposed method.

Figures (5)

• Figure 1: Illustration of GLS correction framework. The DG and UDA methods typically learn the invariant representation across domains. Differently, the proposed GLS correction framework consider both the label shift correction and the probability-aware invariant representation learning. Note that the color of the "Alignment" component is split into orange and black to indicate that both GLS and UDA perform distribution alignment.
• Figure 2: A GLS perspective of CDGE problem. (a) The label probability functions differ in functions support (colored area) and probability values (color degree), resulting the label shift between domains. (b) Conditional distributions of the same gaze label may differ due to factors like identity, background and illuminations.
• Figure 3: Overview of the proposed GLS correction method. (a) Reweighting source label distribution with bivariate Gaussian distribution estimated by pseudo target labels. The reweighted label distribution is utilized in both conditional alignment and task-specific training. (b) Conditional invariant representation learning. Two sets of conditional distribution are map to RKHS and the discrepancy is measured by the PCOD. (c) After label and conditional shift correction, i.e., the GLS correction, predictor trained on source domain can be generalized to target domain. Note that the text color reflects the component each learning objective is associated with.
• Figure 4: Visualization of GLS correction process. For scatter plot (a)(c)(d)(f), label variables are all denoted by '$\bullet$' regardless of value and are distinguished from other domains by colors. In contrast, for t-SNE figure (b) and (e), label values are denoted by color gradients and source domain features are denoted by '$\bullet$' while target features are denoted by '$+$'
• Figure 5: Prediction error under different settings of hyper-parameters.

Theorems & Definitions (1)

• Remark 3.1