Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction

TL;DR

The paper addresses how visual highlighting and cognitive load shape user attention and saliency predictions on webpages. It introduces HCEye, a multi-condition eye-tracking dataset (27 participants, 150 stimuli) combining Absent/Static/Dynamic highlighting with Absent/Low/High cognitive load, analyzed via generalized linear mixed models. Key findings show dynamic highlighting maintains noticeability and directs gaze even under high cognitive load, while static highlighting and cognitive load reduce exploration; saliency models fine-tuned on HCEye with paired dynamic inputs substantially improve prediction accuracy. The work demonstrates the need for temporally aware, state-dependent saliency models to support adaptive UIs and provides a public dataset to catalyze further research in attention under multitasking and dynamic content.

Abstract

Visual highlighting can guide user attention in complex interfaces. However, its effectiveness under limited attentional capacities is underexplored. This paper examines the joint impact of visual highlighting (permanent and dynamic) and dual-task-induced cognitive load on gaze behaviour. Our analysis, using eye-movement data from 27 participants viewing 150 unique webpages reveals that while participants' ability to attend to UI elements decreases with increasing cognitive load, dynamic adaptations (i.e., highlighting) remain attention-grabbing. The presence of these factors significantly alters what people attend to and thus what is salient. Accordingly, we show that state-of-the-art saliency models increase their performance when accounting for different cognitive loads. Our empirical insights, along with our openly available dataset, enhance our understanding of attentional processes in UIs under varying cognitive (and perceptual) loads and open the door for new models that can predict user attention while multitasking.

Figures (7)

• Figure 1: We controlled the size and location of highlighted areas to be equally distributed across images. Left: An example of a small ($11 cm^2$) highlight in location Q2 on a low clutter ($FC = 5.05$) UI. Right: A big $(33.3 cm^2$) highlight in Q4 on a complex UI ($FC = 7.1$). See \ref{['fig:probability highlight seen']} and Supplementary (Supp.) Material for more details.
• Figure 2: Differences in viewing behaviour across the analyzed conditions. White dots and numbers are the Mean, black bars the median.
• Figure 3: The percentage of fixated AOIs (highlighted areas) in each condition.
• Figure 4: The probability that a highlighted area is fixated across different characteristics of the stimulus. We measure the visual clutter of the stimulus using the Feature Congestion metric, while size is operationalized as the area covered by the highlight in cm squared; and location is categorized by dividing the stimulus into four quadrants: Quadrant 1 (Q1) represents the top-left, Q2 the top-right, Q3 signifies the bottom-left, and Q4 the bottom-right.
• Figure 5: Comparing noticeablity of highlighted regions under experimental conditions and their fitted interaction plots. Row 1 compares Time to First Fixation in $ms$, Row 2 the Distance from the Last Fixation (previous focus of attention) in $px$.