SPGen: Stochastic scanpath generation for paintings using unsupervised domain adaptation

TL;DR

SPGen is introduced a novel deep learning model to predict scanpaths the sequence of eye movements when viewers observe paintings, offering a powerful tool to analyze gaze behavior and advance the preservation and appreciation of artistic treasures.

Abstract

Understanding human visual attention is key to preserving cultural heritage We introduce SPGen a novel deep learning model to predict scanpaths the sequence of eye movementswhen viewers observe paintings. Our architecture uses a Fully Convolutional Neural Network FCNN with differentiable fixation selection and learnable Gaussian priors to simulate natural viewing biases To address the domain gap between photographs and artworks we employ unsupervised domain adaptation via a gradient reversal layer allowing the model to transfer knowledge from natural scenes to paintings Furthermore a random noise sampler models the inherent stochasticity of eyetracking data. Extensive testing shows SPGen outperforms existing methods offering a powerful tool to analyze gaze behavior and advance the preservation and appreciation of artistic treasures.

Figures (7)

• Figure 1: General architecture of the proposed method.
• Figure 2: General architecture of the proposed method.
• Figure 3: Calculating congruency of predicted scanpath.
• Figure 4: Distributions of lengths of scanpaths on MIT1003.
• Figure 5: Qualitative predictions on MIT1003.