A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Data

TL;DR

The SOPHIAS (Student Oral Presentation monitoring for Holistic Insights&Analytics using Sensors), a 12-hour multimodal dataset containing recordings of 50 oral presentations, enables the exploration of relationships between multimodal behavioral and physiological signals and presentation performance, and provides a benchmark for developing automated feedback and Multimodal Learning Analytics tools.

Abstract

Oral presentation skills are a critical component of higher education, yet comprehensive datasets capturing real-world student performance across multiple modalities remain scarce. To address this gap, we present SOPHIAS (Student Oral Presentation monitoring for Holistic Insights & Analytics using Sensors), a 12-hour multimodal dataset containing recordings of 50 oral presentations (10-15-minute presentation followed by 5-15-minute Q&A) delivered by 65 undergraduate and master's students at the Universidad Autonoma de Madrid. SOPHIAS integrates eight synchronized sensor streams from high-definition webcams, ambient and webcam audio, eye-tracking glasses, smartwatch physiological sensors, and clicker, keyboard, and mouse interactions. In addition, the dataset includes slides and rubric-based evaluations from teachers, peers, and self-assessments, along with timestamped contextual annotations. The dataset captures presentations conducted in real classroom settings, preserving authentic student behaviors, interactions, and physiological responses. SOPHIAS enables the exploration of relationships between multimodal behavioral and physiological signals and presentation performance, supports the study of peer assessment, and provides a benchmark for developing automated feedback and Multimodal Learning Analytics tools. The dataset is publicly available for research through GitHub and Science Data Bank.

Figures (4)

• Figure 1: Overview of the experimental classroom setup and the synchronized sensing infrastructure deployed during student oral presentations. The figure illustrates all sensing devices and data sources used during data collection: smartwatches worn by the presenter, peer evaluators, teacher, and an external observer, capturing physiological and motion data such as heart rate and inertial signals; eye-tracking glasses worn by the external observer, providing gaze-related measures and first-person video; two external webcams recording the presenter view and the evaluatorsview; ambient and webcam-based audio recordings; mouse, keyboard, and clicker interaction logs; presentation materials in PDF format; contextual event annotations; and rubric-based teacher, peer, and self-assessments collected through the AICoFe system. The spatial arrangement of the presenter, peer evaluators, teacher, and external observer is depicted, together with representative examples of the multimodal data streams captured from each modality. The participants provided consent for the publication of these images
• Figure 2: Pairwise correlations between grades across evaluators (teacher, peers, and self-assessments). Each subplot displays the Pearson correlation coefficient ($r$) and the regression line (in red), with all grades normalized to a 0--10 scale
• Figure 3: Distribution of scores across evaluation sources. All scores were normalized to a 0--10 scale
• Figure 4: Screenshot of the HuMLaS GUI. The participant provided consent for the publication of these image