Evaluating User Experience and Data Quality in Gamified Data Collection for Appearance-Based Gaze Estimation

TL;DR

This work introduces a novel gamified approach for collecting training data in Appearance-based gaze estimation, which uses only a regular camera to estimate human gaze, and assesses the game’s significance on data quality and user experience through a comparative user study.

Abstract

Appearance-based gaze estimation, which uses only a regular camera to estimate human gaze, is important in various application fields. While the technique faces data bias issues, data collection protocol is often demanding, and collecting data from a wide range of participants is difficult. It is an important challenge to design opportunities that allow a diverse range of people to participate while ensuring the quality of the training data. To tackle this challenge, we introduce a novel gamified approach for collecting training data. In this game, two players communicate words via eye gaze through a transparent letter board. Images captured during gameplay serve as valuable training data for gaze estimation models. The game is designed as a physical installation that involves communication between players, and it is expected to attract the interest of diverse participants. We assess the game's significance on data quality and user experience through a comparative user study.

Figures (11)

• Figure 1: This paper proposes a mixed-media gamification system for collecting training data for gaze estimation.
• Figure 2: Hardware overview of the gamified data collection system. Our system consists of A) two webcams, B) a transparent letter board, C) tablet PCs, and D) custom writing boards.
• Figure 3: The flow of the gameplay. Step 1: The questioner first identifies the word, and the answerer observes some clue letters. Step 2: The questioner looks at each letter, and the answerer guesses the rough locations of each letter. Step 3: The answerer guesses the word.
• Figure 4: Flow of our user interface. (1) The questioner and the answerer confirm the target letter, and (2) the questioner triggers the image capture. (3) Once the questioner has approved the captured image, (4) the answerer anticipates and notes the rough position of the letter. (5) When all letters have been captured, the answerer guesses the word and inputs the answer. (6) The system displays the correct answer, and players switch roles and move on to the next word.
• Figure 5: Illustration of the user study settings. The gamified setting was conducted in a public space, where voluntary participants were visitors. The standard-setting experiment was conducted in our laboratory, and participants were recruited directly. We recorded eye tracking data from gamified and standard settings for label accuracy comparison and asked participants to complete a custom survey.