Characterizing Personality from Eye-Tracking: The Role of Gaze and Its Absence in Interactive Search Environments

TL;DR

This work addresses inferring stable personality traits from eye-tracking data during interactive search. It introduces a missing-data-aware multimodal time-series model that treats gaze absence as informative, using raw gaze coordinates, pupil size, and velocity with validity masks and temporal-gap features, processed by a BiLSTM to predict Big Five trait categories. Across five traits, the approach achieves high accuracy in five-fold cross-validation, with Neuroticism and Openness showing the strongest signals, and ablation confirms that missingness information substantially boosts performance. The results demonstrate the feasibility of personality inference from gaze in information-seeking contexts and highlight the value of explicitly modeling missing data, with implications for personalized interactive systems; however, generalization across unseen individuals remains a challenge, calling for larger, more diverse datasets and multimodal extensions.

Abstract

Personality traits influence how individuals engage, behave, and make decisions during the information-seeking process. However, few studies have linked personality to observable search behaviors. This study aims to characterize personality traits through a multimodal time-series model that integrates eye-tracking data and gaze missingness-periods when the user's gaze is not captured. This approach is based on the idea that people often look away when they think, signaling disengagement or reflection. We conducted a user study with 25 participants, who used an interactive application on an iPad, allowing them to engage with digital artifacts from a museum. We rely on raw gaze data from an eye tracker, minimizing preprocessing so that behavioral patterns can be preserved without substantial data cleaning. From this perspective, we trained models to predict personality traits using gaze signals. Our results from a five-fold cross-validation study demonstrate strong predictive performance across all five dimensions: Neuroticism (Macro F1 = 77.69%), Conscientiousness (74.52%), Openness (77.52%), Agreeableness (73.09%), and Extraversion (76.69%). The ablation study examines whether the absence of gaze information affects the model performance, demonstrating that incorporating missingness improves multimodal time-series modeling. The full model, which integrates both time-series signals and missingness information, achieves 10-15% higher accuracy and macro F1 scores across all Big Five traits compared to the model without time-series signals and missingness. These findings provide evidence that personality can be inferred from search-related gaze behavior and demonstrate the value of incorporating missing gaze data into time-series multimodal modeling.

Figures (2)

• Figure 1: Overview of the application interfaces
• Figure 2: Confusion matrices for each personality trait.