Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GameVibe: A Multimodal Affective Game Corpus

Matthew Barthet, Maria Kaselimi, Kosmas Pinitas, Konstantinos Makantasis, Antonios Liapis, Georgios N. Yannakakis

TL;DR

GameVibe introduces a multimodal affect corpus of viewer engagement for FPS game videos, designed to support robust affect modelling. It aggregates 2 hours of synchronized audiovisual game stimuli across 30 FPS titles, with time-continuous engagement traces collected from 20 annotators using RankTrace within a controlled in-lab protocol. The work validates annotator reliability through QA tasks and DTW/KDE-based outlier handling, and releases raw and processed traces along with latent representations derived from pretrained visual and audio models (VideoMAE, MVD, BEATS). This dataset aims to generalize engagement prediction across unseen games and users, enabling cross-game transfer and multimodal analysis for game design and HCI research.

Abstract

As online video and streaming platforms continue to grow, affective computing research has undergone a shift towards more complex studies involving multiple modalities. However, there is still a lack of readily available datasets with high-quality audiovisual stimuli. In this paper, we present GameVibe, a novel affect corpus which consists of multimodal audiovisual stimuli, including in-game behavioural observations and third-person affect traces for viewer engagement. The corpus consists of videos from a diverse set of publicly available gameplay sessions across 30 games, with particular attention to ensure high-quality stimuli with good audiovisual and gameplay diversity. Furthermore, we present an analysis on the reliability of the annotators in terms of inter-annotator agreement.

GameVibe: A Multimodal Affective Game Corpus

TL;DR

GameVibe introduces a multimodal affect corpus of viewer engagement for FPS game videos, designed to support robust affect modelling. It aggregates 2 hours of synchronized audiovisual game stimuli across 30 FPS titles, with time-continuous engagement traces collected from 20 annotators using RankTrace within a controlled in-lab protocol. The work validates annotator reliability through QA tasks and DTW/KDE-based outlier handling, and releases raw and processed traces along with latent representations derived from pretrained visual and audio models (VideoMAE, MVD, BEATS). This dataset aims to generalize engagement prediction across unseen games and users, enabling cross-game transfer and multimodal analysis for game design and HCI research.

Abstract

As online video and streaming platforms continue to grow, affective computing research has undergone a shift towards more complex studies involving multiple modalities. However, there is still a lack of readily available datasets with high-quality audiovisual stimuli. In this paper, we present GameVibe, a novel affect corpus which consists of multimodal audiovisual stimuli, including in-game behavioural observations and third-person affect traces for viewer engagement. The corpus consists of videos from a diverse set of publicly available gameplay sessions across 30 games, with particular attention to ensure high-quality stimuli with good audiovisual and gameplay diversity. Furthermore, we present an analysis on the reliability of the annotators in terms of inter-annotator agreement.
Paper Structure (9 sections, 5 figures, 1 table)

This paper contains 9 sections, 5 figures, 1 table.

Table of Contents

  1. Participant recruitment.
  2. Laboratory settings.
  3. Research ethics.
  4. Survey data.
  5. Annotation protocol and tools.
  6. Collected annotation data.
  7. Quality assurance and reproducibility of the GameVibe affect annotations.
  8. Audiovisual stimuli data processing.
  9. Affect processing methodology.

Figures (5)

  • Figure 1: High-level overview of the experimental protocol used for data collection in this study.
  • Figure 2: Details of the GameVibe audiovisual stimuli. The figure illustrates screenshots from the 30 different FPS games that are annotated for engagement (top left), the release date of each title in ascending order (bottom left), the names of each FPS title (top right) and the proportion of games in terms of game modes, game styles and game designs (bottom right).
  • Figure 3: An example of outlier detection using the game Wolfenstein 3D (Apogee Software, 1992) from Session 1. Figure 3(a) depicts the unfiltered traces from five annotators. The frequency distribution can be seen in Figure 3(b), depicting the signals' KDE score used to create the outlier filters. The red and orange lines depict the 90% filter and 80% filter thresholds, respectively. Annotators to the right of both filters are considered inliers and are not removed. Figure 3(c) shows the first filter applied removing one outlier (P3), and Figure 3(d) shows the strictest filter applied removing two outliers (P3, P4).
  • Figure 4: Diagram overview of the dataset and its structure.
  • Figure 5: GameVibe's post-processing pipeline, visualising the transformed annotations and the output files of each stage.