Collaborative Problem Solving in Mixed Reality: A Study on Visual Graph Analysis

TL;DR

This study evaluates collaborative problem solving in visuospatial graph tasks conducted in mixed reality, comparing ad hoc pairs, individuals, and nominal pairs. Using two graph tasks and a within-subject task-instance design across 72 participants in two countries, the authors quantify accuracy, completion time, and cognitive load to distinguish collaboration from aggregation. They introduce task instance complexity to analyze how signal and noise components shape performance, finding that ad hoc pairs achieve higher accuracy but at a time cost, while nominal pairs serve as a crucial benchmark. The findings imply that MR environments do not automatically enhance collaboration and that benchmarking against nominal groups is essential for valid CVE assessments, with broader implications for designing collaborative visualizations and interfaces.

Abstract

Problem solving is a composite cognitive process, invoking a number of systems and subsystems, such as perception and memory. Individuals may form collectives to solve a given problem together, in collaboration, especially when complexity is thought to be high. To determine if and when collaborative problem solving is desired, we must quantify collaboration first. For this, we investigate the practical virtue of collaborative problem solving. Using visual graph analysis, we perform a study with 72 participants in two countries and three languages. We compare ad hoc pairs to individuals and nominal pairs, solving two different tasks on graphs in visuospatial mixed reality. The average collaborating pair does not outdo its nominal counterpart, but it does have a significant trade-off against the individual: an ad hoc pair uses 1.46 more time to achieve 4.6 higher accuracy. We also use the concept of task instance complexity to quantify differences in complexity. As task instance complexity increases, these differences largely scale, though with two notable exceptions. With this study we show the importance of using nominal groups as benchmark in collaborative virtual environments research. We conclude that a mixed reality environment does not automatically imply superior collaboration.

Figures (7)

• Figure 1: Participants solving visuospatial tasks on graphs in mixed reality. Participants who were assigned to ad hoc pairs solved the tasks collaboratively, while remaining participants solved these on their own, either as individuals or in a nominal pair.
• Figure 2: Path nodes $w\in u...v$ (yellow) are enclosed by their minimum volume ellipsoid $E_3$ (red), as seen in the small view. Nodes and edges not part of $u...v$, which lie wholly or partly in $E_3$, are then counted as noise (cyan).
• Figure 3: a) Absolute difference in accuracy, paired, between group types. Error bars mark significant difference from the null, based on bootstrapped $95$% CI. b), c) Average completion time, seconds, given by group type. Horizontal bars (top) indicate significance after post hoc correction.
• Figure 4: Completion time by task instance complexity for ad hoc and nominal pairs. As signal instance complexity increases (left), nominal pairs tend to slow down more than ad hoc pairs. For increases in noise instance complexity (right), the difference between pair types remains largely unchanged.
• Figure 5: Average probability of high cognitive load for total noise instance complexity from the sample mean of $194m$.