Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MoCoMR: A Collaborative MR Simulator with Individual Behavior Modeling

Diana Romero, Fatima Anwar, Salma Elmalaki

TL;DR

MoCoMR introduces a data-driven MR collaboration simulator that synthesizes multimodal behavior—speaking, gaze, and locomotion—from real-world data collected across 12 groups of 4 participants. By clustering interaction types with Gaussian Mixture Models and generating sequences through modality-specific stochastic processes, it produces synthetic logs and sociograms for group dynamics. The approach is validated against real data, showing strong structural fidelity in group interactions and task-related metrics, with some discrepancies in spatial and attention patterns and limited predictability for some task outcomes. The accompanying API enables researchers to configure experiments, run simulations, and export sociograms and metrics, offering a scalable tool to advance human-centered MR design and evaluation.

Abstract

Studying collaborative behavior in Mixed Reality (MR) often requires extensive, challenging data collection. This paper introduces MoCoMR, a novel simulator designed to address this by generating synthetic yet realistic collaborative MR data. MoCoMR captures individual behavioral modalities such as speaking, gaze, and locomotion during a collaborative image-sorting task with 48 participants to identify distinct behavioral patterns. MoCoMR simulates individual actions and interactions within a virtual space, enabling researchers to investigate the impact of individual behaviors on group dynamics and task performance. This simulator facilitates the development of more effective and human-centered MR applications by providing insights into user behavior and interaction patterns. The simulator's API allows for flexible configuration and data analysis, enabling researchers to explore various scenarios and generate valuable insights for optimizing collaborative MR experiences.

MoCoMR: A Collaborative MR Simulator with Individual Behavior Modeling

TL;DR

MoCoMR introduces a data-driven MR collaboration simulator that synthesizes multimodal behavior—speaking, gaze, and locomotion—from real-world data collected across 12 groups of 4 participants. By clustering interaction types with Gaussian Mixture Models and generating sequences through modality-specific stochastic processes, it produces synthetic logs and sociograms for group dynamics. The approach is validated against real data, showing strong structural fidelity in group interactions and task-related metrics, with some discrepancies in spatial and attention patterns and limited predictability for some task outcomes. The accompanying API enables researchers to configure experiments, run simulations, and export sociograms and metrics, offering a scalable tool to advance human-centered MR design and evaluation.

Abstract

Studying collaborative behavior in Mixed Reality (MR) often requires extensive, challenging data collection. This paper introduces MoCoMR, a novel simulator designed to address this by generating synthetic yet realistic collaborative MR data. MoCoMR captures individual behavioral modalities such as speaking, gaze, and locomotion during a collaborative image-sorting task with 48 participants to identify distinct behavioral patterns. MoCoMR simulates individual actions and interactions within a virtual space, enabling researchers to investigate the impact of individual behaviors on group dynamics and task performance. This simulator facilitates the development of more effective and human-centered MR applications by providing insights into user behavior and interaction patterns. The simulator's API allows for flexible configuration and data analysis, enabling researchers to explore various scenarios and generate valuable insights for optimizing collaborative MR experiences.

Paper Structure

This paper contains 14 sections, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work and Contribution
  3. Methodology
  4. Data Collection
  5. Individual Behavior Modeling
  6. Individual Behavior Simulation and Group Interaction Modeling
  7. MoCoMR API Design
  8. Evaluation
  9. Individual Speaking Behaviors
  10. Individual Gaze Behaviors
  11. Individual Locomotion Behaviors
  12. Individual Object Interaction Behaviors
  13. MoCoMR Simulated Data Evaluation
  14. Discussion & Conclusion

Figures (8)

  • Figure 1: Overview of the MoCoMR design pipeline. Features from various interaction types are used to identify emergent individual behaviors through Gaussian Mixture Model clustering. The identified behaviors serve as parameters to simulate individual types in our proposed MoCoMR simulator.
  • Figure 2: Distribution of mean speaking duration by cluster.
  • Figure 3: Real-world and simulated mean speaking features.
  • Figure 4: Distribution of mean gaze duration by cluster.
  • Figure 5: Real-world and simulated mean gaze features.
  • ...and 3 more figures