Swarm manipulation: An efficient and accurate technique for multi-object manipulation in virtual reality

TL;DR

A novel Swarm Manipulation interaction technique is introduced and compared with two baseline techniques: Virtual Hand and Controller (ray-casting) and Swarm Manipulation, which yielded superior performance, with significantly faster speeds in most conditions across the three tasks.

Abstract

The theory of swarm control shows promise for controlling multiple objects, however, scalability is hindered by cost constraints, such as hardware and infrastructure. Virtual Reality (VR) can overcome these limitations, but research on swarm interaction in VR is limited. This paper introduces a novel Swarm Manipulation interaction technique and compares it with two baseline techniques: Virtual Hand and Controller (ray-casting). We evaluated these techniques in a user study ($N$ = 12) in three tasks (selection, rotation, and resizing) across five conditions. Our results indicate that Swarm Manipulation yielded superior performance, with significantly faster speeds in most conditions across the three tasks. It notably reduced resizing size deviations but introduced a trade-off between speed and accuracy in the rotation task. Additionally, we conducted a follow-up user study ($N$ = 6) using Swarm Manipulation in two complex VR scenarios and obtained insights through semi-structured interviews, shedding light on optimized swarm control mechanisms and perceptual changes induced by this interaction paradigm. These results demonstrate the potential of the Swarm Manipulation technique to enhance the usability and user experience in VR compared to conventional manipulation techniques. In future studies, we aim to understand and improve swarm interaction via internal swarm particle cooperation.

Figures (12)

• Figure 1: We introduce the Swarm Manipulation technique in Virtual Reality (VR) and compare it with two conventional manipulation techniques: Hand and Controller (Ray-Casting). We evaluate these techniques based on three tasks: (1) Selection, (2) Rotation, and (3) Resizing. During object selection, the swarm particles transition from their original color to blue.
• Figure 2: The figure illustrates the Swarm Hand concept consisting of two key components. The first component, the Swarm Hand, is composed of a swarm of particles, controlled by users through hand gestures. The second component, the non-dominant hand depicted as the left hand (L) in the figure, enables adjustment of distribution levels. This feature determines the size of the Swarm Hand and allows users to grasp or select multiple virtual objects based on their preferences.
• Figure 3: There are twelve objects that should be manipulated during the study. The selected objects are highlighted in red, while the target object is highlighted in green.
• Figure 4: A screenshot of the Swarm Manipulation technique in VR. It showcases the non-dominant hand with a distribution bar encircling the left wrist, while the dominant hand interacts with the swarm. In the field of view, twelve targets are positioned ahead.
• Figure 5: Mean task-completion times for each technique across five conditions of Task 1. Significant differences between conditions are annotated above the bars, with '*', '**', and '***' indicating significance levels at $p < 0.05$, $p < 0.01$, and $p < 0.001$, respectively.