Polarization-resolved imaging improves eye tracking

TL;DR

Polarization-resolved near-infrared imaging adds a polarization contrast to standard eye-tracking imaging by measuring AoLP and DoLP from ocular tissues. The authors implement polarization-enabled eye tracking (PET) using a polarization-filter-array camera paired with a linearly polarized NIR illuminator, enabling per-pixel AoLP/DoLP maps alongside intensity and revealing scleral textures and corneal patterns not visible in intensity images. In a study with 346 participants, CNNs trained on PET data reduce the population tail gaze error, quantified as $U_{50}E_{95}$, by about $0.12$–$0.23$ degrees (roughly 10.3%–15.9%) relative to capacity-matched intensity baselines, under nominal conditions and non-ideal factors such as eyelid occlusion, eye-relief shifts, and pupil-size changes. The results persist across different camera placements, demonstrate week-long stability of polarization features, and support PET as a simple, robust sensing modality for wearable eye-tracking.

Abstract

Polarization-resolved near-infrared imaging adds a useful optical contrast mechanism to eye tracking by measuring the polarization state of light reflected by ocular tissues in addition to its intensity. In this paper we demonstrate how this contrast can be used to enable eye tracking. Specifically, we demonstrate that a polarization-enabled eye tracking (PET) system composed of a polarization--filter--array camera paired with a linearly polarized near-infrared illuminator can reveal trackable features across the sclera and gaze-informative patterns on the cornea, largely absent in intensity-only images. Across a cohort of 346 participants, convolutional neural network based machine learning models trained on data from PET reduced the median 95th-percentile absolute gaze error by 10--16\% relative to capacity-matched intensity baselines under nominal conditions and in the presence of eyelid occlusions, eye-relief changes, and pupil-size variation. These results link light--tissue polarization effects to practical gains in human--computer interaction and position PET as a simple, robust sensing modality for future wearable devices.

Figures (7)

• Figure 1: PET images with good cornea visibility across diverse subjects. The intensity images and the polarization heatmaps are shown in (A) and (B), respectively. Among the subjects, 4 subjects were wearing contact lenses during the data collection. And 2 participants had undergone laser eye surgeries (i.e., Subject 3 had SMILE, Subject 10 had LASIK)
• Figure 2: Polarization-resolved features over 4 weeks in one subject. (A) Intensity images. (B) DoLP. (C) AoLP. The persistence of polarization-derived scleral texture and corneal patterns across sessions support robust, personalized eye tracking and long‑term calibration.
• Figure 3: Intensity-only versus polarization-enabled eye tracking under matched acquisition and training. From identical raw polarization recordings, we formed (i) pseudo‑intensity inputs (superpixel averaging) and (ii) four‑channel polarization inputs $(0^{\circ}, 45^{\circ}, 90^{\circ}, 135^{\circ})$, trained capacity‑matched models, and compared population‑wise $U_{50}E_{95}$ across percentiles. A–F, Median PET–intensity difference (blue) with 90% bootstrapped confidence envelopes for higher/lower temporal placements under three test conditions: nominal, eye‑relief change (no re‑calibration), and pupil‑size change (no re‑calibration). Shaded bands annotate regimes of lower error for each modality. G–S, Representative input pairs (pseudo‑intensity vs polarization‑resolved) by condition.
• Figure 4: Polarization-resolved imaging of the human eye. Panels show (a--d) the four reconstructed linear polarization channels at $0^\circ$, $45^\circ$, $90^\circ$, and $135^\circ$ (grayscale); (e) total intensity $I = S_0/4$; (f) degree of linear polarization $\mathrm{DoLP} = \sqrt{S_1^2 + S_2^2}/(S_0 + \varepsilon)$ clipped to $[0, 1]$; (g) angle of linear polarization $\mathrm{AoLP} = \tfrac{1}{2}\,\arctan2(S_2, S_1)$ (radians); and (h) an HSV composite where hue encodes AoLP, saturation scales with DoLP, and value is the gamma-corrected intensity. All panels are rotated by $180^\circ$ and contrast-normalized for visualization.
• Figure 5: Data collection hardware. (A) PET subsystem and polarized 850 nm flood illuminator integrated in an NFF benchtop station; polarization-sensitive cameras and a single-camera/illuminator PET concept. (B) Photograph of the custom in-house station with chinrest and adjustable kinematics for ER/IPD coverage.