Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Study of Performance and Interaction Patterns in Hand and Tangible Interaction in Tabletop Mixed Reality

Carlos Mosquera, Neven Elsayed, Ernst Kruijff, Joseph Newman, Eduardo Veas

TL;DR

This work compares hand-based and tangible input for manipulating virtual objects along $4$DoF on real surfaces in mixed reality, introducing a knob-shaped tangible proxy designed for tabletop stability. Through a pilot study to identify an optimal scaling method and a main study across Move/Rotate, Scale, and Combined tasks, the authors show tangible input generally offers higher precision and competitive or faster performance for many $4$DoF tasks, though modal switching in scaling can add cognitive load. Across tasks, Tangible outperforms Hand in most metrics, with rotation-based scaling (ROT) proving particularly effective, and phrasing that the tabletop constraint and reduced effective DoF contribute to improved control. These findings inform designers about trade-offs between hand-only and tangible interactions in MR tabletop applications, suggesting tangibles for precision-critical tasks while noting potential complexity in multi-modal scaling scenarios.

Abstract

This paper presents a comprehensive study of virtual 3D object manipulation along 4DoF on real surfaces in mixed reality (MR), using hand-based and tangible interactions. A custom cylindrical tangible proxy leverages affordances of physical knobs and tabletop support for stable input. We evaluate both modalities across isolated tasks (2DoF translation, 1DoF rotation scaling), semicombined (3DoF translation rotation), and full 4DoF compound manipulation. We offer analyses of hand interactions, tangible interactions, and their comparison in MR tasks. For hand interactions, compound tasks required repetitive corrections, increasing completion times yet surprisingly, rotation errors were smaller in compound tasks than in rotation only tasks. Tangible interactions exhibited significantly larger errors in translation, rotation, and scaling during compound tasks compared to isolated tasks. Crucially, tangible interactions outperformed hand interactions in precision, likely due to tabletop support and constrained 4DoF design. These findings inform designers opting for hand-only interaction (highlighting tradeoffs in compound tasks) and those leveraging tangibles (emphasizing precision gains despite compound-task challenges).

A Study of Performance and Interaction Patterns in Hand and Tangible Interaction in Tabletop Mixed Reality

TL;DR

This work compares hand-based and tangible input for manipulating virtual objects along DoF on real surfaces in mixed reality, introducing a knob-shaped tangible proxy designed for tabletop stability. Through a pilot study to identify an optimal scaling method and a main study across Move/Rotate, Scale, and Combined tasks, the authors show tangible input generally offers higher precision and competitive or faster performance for many DoF tasks, though modal switching in scaling can add cognitive load. Across tasks, Tangible outperforms Hand in most metrics, with rotation-based scaling (ROT) proving particularly effective, and phrasing that the tabletop constraint and reduced effective DoF contribute to improved control. These findings inform designers about trade-offs between hand-only and tangible interactions in MR tabletop applications, suggesting tangibles for precision-critical tasks while noting potential complexity in multi-modal scaling scenarios.

Abstract

This paper presents a comprehensive study of virtual 3D object manipulation along 4DoF on real surfaces in mixed reality (MR), using hand-based and tangible interactions. A custom cylindrical tangible proxy leverages affordances of physical knobs and tabletop support for stable input. We evaluate both modalities across isolated tasks (2DoF translation, 1DoF rotation scaling), semicombined (3DoF translation rotation), and full 4DoF compound manipulation. We offer analyses of hand interactions, tangible interactions, and their comparison in MR tasks. For hand interactions, compound tasks required repetitive corrections, increasing completion times yet surprisingly, rotation errors were smaller in compound tasks than in rotation only tasks. Tangible interactions exhibited significantly larger errors in translation, rotation, and scaling during compound tasks compared to isolated tasks. Crucially, tangible interactions outperformed hand interactions in precision, likely due to tabletop support and constrained 4DoF design. These findings inform designers opting for hand-only interaction (highlighting tradeoffs in compound tasks) and those leveraging tangibles (emphasizing precision gains despite compound-task challenges).

Paper Structure

This paper contains 42 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. TUIs on tabletops
  4. TUIs for object manipulation in MR
  5. Study Design
  6. 4DoF Use case
  7. Tangible prop design
  8. Apparatus
  9. Interaction Design
  10. HAND Interactor
  11. TAN Interactor
  12. Pilot Study: Optimal Scaling Method
  13. Pilot Study Design
  14. Pilot Study Results
  15. Main Study: Full Manipulation Pipeline
  16. ...and 27 more sections

Figures (3)

  • Figure 1: Scaling Pilot Results: Effect of interaction technique on performance. Left: Completion Time, Right: Error
  • Figure 2: Results for HAND and TAN: (Left) Pairwise test results for metrics across tasks, (Middle) Sequential Analysis for Combined task from a representative user, (Right) Spatial pattern of interactors from a representative user in Only-Move
  • Figure 3: NASA-TLX for the combined task