UpStory: the Uppsala Storytelling dataset

TL;DR

This work presents UpStory — the Uppsala Storytelling dataset: a novel dataset of naturalistic dyadic interactions between primary school aged children, with an experimental manipulation of rapport, and confirms the informative power of the UpStory dataset by establishing baselines for the prediction of rapport.

Abstract

Friendship and rapport play an important role in the formation of constructive social interactions, and have been widely studied in educational settings due to their impact on student outcomes. Given the growing interest in automating the analysis of such phenomena through Machine Learning (ML), access to annotated interaction datasets is highly valuable. However, no dataset on dyadic child-child interactions explicitly capturing rapport currently exists. Moreover, despite advances in the automatic analysis of human behaviour, no previous work has addressed the prediction of rapport in child-child dyadic interactions in educational settings. We present UpStory -- the Uppsala Storytelling dataset: a novel dataset of naturalistic dyadic interactions between primary school aged children, with an experimental manipulation of rapport. Pairs of children aged 8-10 participate in a task-oriented activity: designing a story together, while being allowed free movement within the play area. We promote balanced collection of different levels of rapport by using a within-subjects design: self-reported friendships are used to pair each child twice, either minimizing or maximizing pair separation in the friendship network. The dataset contains data for 35 pairs, totalling 3h 40m of audio and video recordings. It includes two video sources covering the play area, as well as separate voice recordings for each child. An anonymized version of the dataset is made publicly available, containing per-frame head pose, body pose, and face features; as well as per-pair information, including the level of rapport. Finally, we provide ML baselines for the prediction of rapport.

Figures (13)

• Figure 1: Friendship network of the Year 2 cohort. Vertex color indicates class (labeled randomly as A, C, D for anonymity). Edge color and shape indicate type of connection (one-way or mutual). The light gray loop corresponds to a child who nominated themselves.
• Figure 2: Friendship network of the Year 3 cohort. All children belonged to the same class (labeled randomly as B; shown in purple). Edge color and shape indicate type of connection (one-way or mutual).
• Figure 3: Example of a playing board, showing face-up and face-down cards. Each row corresponds to one setting. Top to bottom: halloween monsters, hiking, fantasy. Each column corresponds to a card type. Left to right: location, character, character, object.
• Figure 4: IOS questionnaire aron1992inclusion, used to measure the closeness of each participant with their best friend, a "bad guy" from fictional media, and their partner during the game. Measured after the interaction.
• Figure 5: SAM questionnaire bradley1994measuring, used to measure the Valence (top row, left to right), Arousal (middle row, right to left), and Dominance (bottom row, left to right) dimensions of emotion. Measured before and after the interaction.