Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AdSight: Scalable and Accurate Quantification of User Attention in Multi-Slot Sponsored Search

Mario Villaizán-Vallelado, Matteo Salvatori, Kayhan Latifzadeh, Antonio Penta, Luis A. Leiva, Ioannis Arapakis

TL;DR

AdSight introduces a Transformer-based Seq2Seq model to quantify user attention in multi-slot SERPs using mouse cursor trajectories and eye-tracking ground truth. It encodes cursor motion as time-series and leverages slot metadata and auxiliary slots within a unified encoder-decoder architecture to predict per-slot attention metrics for regression and binary slot-notice classification. The approach outperforms strong baselines across multiple representations, with time-series cursors and Transformer embeddings delivering the strongest results and demonstrating robust generalization to varying layouts. The work provides a scalable, low-cost method to infer attention patterns in complex SERP layouts, enabling improved ad placement and user experience optimization.

Abstract

Modern Search Engine Results Pages (SERPs) present complex layouts where multiple elements compete for visibility. Attention modelling is crucial for optimising web design and computational advertising, whereas attention metrics can inform ad placement and revenue strategies. We introduce AdSight, a method leveraging mouse cursor trajectories to quantify in a scalable and accurate manner user attention in multi-slot environments like SERPs. AdSight uses a novel Transformer-based sequence-to-sequence architecture where the encoder processes cursor trajectory embeddings, and the decoder incorporates slot-specific features, enabling robust attention prediction across various SERP layouts. We evaluate our approach on two Machine Learning tasks: (1) regression, to predict fixation times and counts; and (2) classification, to determine some slot types were noticed. Our findings demonstrate the model's ability to predict attention with unprecedented precision, offering actionable insights for researchers and practitioners.

AdSight: Scalable and Accurate Quantification of User Attention in Multi-Slot Sponsored Search

TL;DR

AdSight introduces a Transformer-based Seq2Seq model to quantify user attention in multi-slot SERPs using mouse cursor trajectories and eye-tracking ground truth. It encodes cursor motion as time-series and leverages slot metadata and auxiliary slots within a unified encoder-decoder architecture to predict per-slot attention metrics for regression and binary slot-notice classification. The approach outperforms strong baselines across multiple representations, with time-series cursors and Transformer embeddings delivering the strongest results and demonstrating robust generalization to varying layouts. The work provides a scalable, low-cost method to infer attention patterns in complex SERP layouts, enabling improved ad placement and user experience optimization.

Abstract

Modern Search Engine Results Pages (SERPs) present complex layouts where multiple elements compete for visibility. Attention modelling is crucial for optimising web design and computational advertising, whereas attention metrics can inform ad placement and revenue strategies. We introduce AdSight, a method leveraging mouse cursor trajectories to quantify in a scalable and accurate manner user attention in multi-slot environments like SERPs. AdSight uses a novel Transformer-based sequence-to-sequence architecture where the encoder processes cursor trajectory embeddings, and the decoder incorporates slot-specific features, enabling robust attention prediction across various SERP layouts. We evaluate our approach on two Machine Learning tasks: (1) regression, to predict fixation times and counts; and (2) classification, to determine some slot types were noticed. Our findings demonstrate the model's ability to predict attention with unprecedented precision, offering actionable insights for researchers and practitioners.
Paper Structure (35 sections, 3 figures, 7 tables)

This paper contains 35 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. User Modelling
  4. Inferring User Interest
  5. Inferring Visual Attention
  6. Inferring Personally Identifiable Information
  7. Sponsored Search
  8. User study
  9. Participants
  10. Design
  11. Procedure
  12. Apparatus
  13. Data representation
  14. Cursor Data
  15. Multivariate Time Series
  16. ...and 20 more sections

Figures (3)

  • Figure 1: Visual representations of mouse movements to train the models, all including slot placeholders.
  • Figure 2: Examples of Google with multi-slot layout. Slot categories are colour-coded ( direct-top slots, direct-right slots, organic slots on top, and organic slots on bottom). Cursor movements are visualized as continuous black lines, while eye-tracking events are represented by filled circles (shown in black if within auxiliary slots or red if within standard slots). Circle radius is proportional to event duration.
  • Figure 3: Comparison of the baseline (\ref{['fig:baseline_model']}) and approach (\ref{['fig:seq2seq_model']}). The baseline embeds cursor data into a dense latent space, using an adapted to the maximum slots per document, with metadata-based filtering. The model projects cursor data and metadata into a shared latent space, leveraging a Transformer Encoder-Decoder for sequence relationships. Regression outputs estimate quantities (/); classification aggregates scores by type and applies a sigmoid for probabilities.