Attention-Aware Visualization: Tracking and Responding to User Perception Over Time

TL;DR

This paper introduces Attention-Aware Visualization (AAV), a framework that tracks a viewer's attention on a visualization over time and feeds that information back to adjust the visualization. It provides two concrete implementations: a 2D data-agnostic version for web-based visuals (eyetracker/mouse-based gaze) and a 3D data-aware version using a stencil buffer to map attention to individual marks in immersive environments. The authors present a common attention model with global and short-term maps, and explore revisualization strategies (heatmaps, contours, emphasis) alongside triggering mechanisms (Always-on, Explicit, Implicit). A qualitative evaluation across both 2D and 3D contexts demonstrates the potential of AAV to guide exploration and support introspection, while also highlighting design trade-offs and limitations such as possible self-reinforcing attention loops and reliance on gaze-tracking availability. The work points to future directions in multimodal, context-aware visualization systems and learning-based attention modeling to enhance immersive analytics and education.

Abstract

We propose the notion of Attention-Aware Visualizations (AAVs) that track the user's perception of a visual representation over time and feed this information back to the visualization. Such context awareness is particularly useful for ubiquitous and immersive analytics where knowing which embedded visualizations the user is looking at can be used to make visualizations react appropriately to the user's attention: for example, by highlighting data the user has not yet seen. We can separate the approach into three components: (1) measuring the user's gaze on a visualization and its parts; (2) tracking the user's attention over time; and (3) reactively modifying the visual representation based on the current attention metric. In this paper, we present two separate implementations of AAV: a 2D data-agnostic method for web-based visualizations that can use an embodied eyetracker to capture the user's gaze, and a 3D data-aware one that uses the stencil buffer to track the visibility of each individual mark in a visualization. Both methods provide similar mechanisms for accumulating attention over time and changing the appearance of marks in response. We also present results from a qualitative evaluation studying visual feedback and triggering mechanisms for capturing and revisualizing attention.

Figures (8)

• Figure 1: Triggering strategies. Overview of the different triggering strategies for our attention-aware visualization interventions.
• Figure 2: AAV2D picture framing metaphor. AAV2D provides several methods for revisualizing attention as an (a) Overlay, either on the Mount that holds the visualization or on a (b) Minimap around the (c) decorated Border that revisualizes attention.
• Figure 3: AAV2D overlay (glaze) revisualizations. The three primary revisualization mechanisms include (a) heatmap, (b) contour plot and (c) mesh. The colors shown in the figure are tailored to fit the corresponding mounted visualization, which in this case is John Snow's Cholera Map.
• Figure 4: AAV2D border revisualizations. AAV2D provides several methods for revisualizing attention on the decorative border of the element, each of which can be tailored to fit the mounted visualization.
• Figure 5: AAV3D revisualizations. AAV3D provides several methods for revisualizing attention on the visual marks themselves.