What Sensors See, What People Feel: An Exploratory Study of Subjective Collaboration Perception in Mixed Reality
Yasra Chandio, Diana Romero, Salma Elmalaki, Fatima Anwar
TL;DR
This work addresses the challenge of inferring subjective collaboration experience from sensor data in mixed reality. It proposes the Sensor-to-Subjective (S2S) Mapping Framework to align subjective perceptions, sensor-derived indicators, and emergent task outcomes. An exploratory study with 48 participants across 12 MR groups examines gaze, speech, and proximity signals during a collaborative image-sorting task, finding robust group-level links between shared gaze, proximity, and perceived collaboration as well as task performance, while subjective states like presence show weaker sensor mappings. The results support the value of interpretive anchors for human-centered sensing in MR and highlight implications for designing cooperative MR systems that transparently reflect user experience, while also noting the need for multimodal and hybrid approaches to better capture internal states.
Abstract
Mixed Reality (MR) enables rich, embodied collaboration; however, it is uncertain whether sensor- and system-logged behavioral signals capture how users experience that collaboration. This disconnect stems from a fundamental gap. Behavioral signals are observable and continuous, while collaboration is interpreted subjectively and shaped by internal states like presence, cognitive availability, and social awareness. Our core insight is that sensor signals serve as observable manifestations of subjective experiences in MR collaboration, and they can be captured through sensor data such as shared gaze, speech, spatial movement, and other system-logged performance metrics. We propose the Sensor-to-Subjective (S2S) Mapping Framework, a conceptual model that links observable interaction patterns to users' subjective perceptions of collaboration and internal cognitive states through sensor-based indicators and task performance metrics. To evaluate this model, we conducted an exploratory study with 48 participants across 12 MR groups engaged in a collaborative image-sorting task. Our findings show a correlation between sensed behavior and perceived collaboration, particularly through shared attention and proximity.