Real-Time Multimodal Data Collection Using Smartwatches and Its Visualization in Education

TL;DR

The paper tackles the challenge of real-time, scalable multimodal data collection in education by introducing two complementary tools: Watch-DMLT for synchronized smartwatch data capture and ViSeDOPS for visualization and analysis. It demonstrates a classroom deployment with 65 students and up to 16 devices, capturing heart rate, motion, gaze, video, and contextual annotations to enable fine-grained analysis of engagement and performance. The work shows the feasibility and utility of end-to-end data acquisition and visualization in real learning environments, addressing barriers to the adoption of Multimodal Learning Analytics. Together, these tools lay the groundwork for broader application in authentic educational settings and pave the way for more sophisticated analyses and predictive insights.

Abstract

Wearable sensors, such as smartwatches, have become increasingly prevalent across domains like healthcare, sports, and education, enabling continuous monitoring of physiological and behavioral data. In the context of education, these technologies offer new opportunities to study cognitive and affective processes such as engagement, attention, and performance. However, the lack of scalable, synchronized, and high-resolution tools for multimodal data acquisition continues to be a significant barrier to the widespread adoption of Multimodal Learning Analytics in real-world educational settings. This paper presents two complementary tools developed to address these challenges: Watch-DMLT, a data acquisition application for Fitbit Sense 2 smartwatches that enables real-time, multi-user monitoring of physiological and motion signals; and ViSeDOPS, a dashboard-based visualization system for analyzing synchronized multimodal data collected during oral presentations. We report on a classroom deployment involving 65 students and up to 16 smartwatches, where data streams including heart rate, motion, gaze, video, and contextual annotations were captured and analyzed. Results demonstrate the feasibility and utility of the proposed system for supporting fine-grained, scalable, and interpretable Multimodal Learning Analytics in real learning environments.

Figures (3)

• Figure 1: Data communication architecture between smartwatch, phone, and server.
• Figure 2: Overview of the ViSeDOPS system architecture.
• Figure 3: Some visualizations extracted from ViSeDOPS.