Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reimagining Wearable AR Gesture Design: Physical Therapy Reasoning in Everyday Contexts

Wei Wu, Binyan Xu, Soonhyeon Kweon, Yujie Wang, Leanne Chukoskie, Casper Harteveld

Abstract

Lightweight augmented reality (AR) glasses are increasingly entering everyday use, extending interaction design beyond short, isolated sessions. However, most existing gesture vocabularies are inherited from VR headsets or early AR goggles. These systems tend to prioritize recognizer accuracy while overlooking fatigue, sustainability, and social legibility in daily contexts. To address this gap, we collaborated with physical therapists (PTs) to reimagine gesture design for everyday AR, drawing on their expertise in safe and sustainable movement. Through a review of 104 AR applications, we identified 15 common gesture intents and implemented an on-device gesture generator. Ten licensed physical therapists, with an average of 14.8 years of professional experience, then shaped these gesture intents through three iterative stages: unaided gesture performance, PT-guided gesture substitution, and stage-aware card sorting. This work contributes (1) a PT-informed gesture translation method, (2) the Everyday-AR Golden Ergonomic Canvas, and (3) a stage-aware social legibility framework that illustrates how gesture suitability shifts with social readability. Together, these contributions provide a recognizer-agnostic reference framework for designing sustainable and socially coherent gesture vocabularies for lightweight AR glasses.

Reimagining Wearable AR Gesture Design: Physical Therapy Reasoning in Everyday Contexts

Abstract

Lightweight augmented reality (AR) glasses are increasingly entering everyday use, extending interaction design beyond short, isolated sessions. However, most existing gesture vocabularies are inherited from VR headsets or early AR goggles. These systems tend to prioritize recognizer accuracy while overlooking fatigue, sustainability, and social legibility in daily contexts. To address this gap, we collaborated with physical therapists (PTs) to reimagine gesture design for everyday AR, drawing on their expertise in safe and sustainable movement. Through a review of 104 AR applications, we identified 15 common gesture intents and implemented an on-device gesture generator. Ten licensed physical therapists, with an average of 14.8 years of professional experience, then shaped these gesture intents through three iterative stages: unaided gesture performance, PT-guided gesture substitution, and stage-aware card sorting. This work contributes (1) a PT-informed gesture translation method, (2) the Everyday-AR Golden Ergonomic Canvas, and (3) a stage-aware social legibility framework that illustrates how gesture suitability shifts with social readability. Together, these contributions provide a recognizer-agnostic reference framework for designing sustainable and socially coherent gesture vocabularies for lightweight AR glasses.
Paper Structure (50 sections, 16 figures, 4 tables)

This paper contains 50 sections, 16 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Background: From Enterprise AR to Everyday AR Interaction
  4. Methods for Gesture Design
  5. Embodied Interaction and Soma-Based Perspectives on Movement
  6. Ergonomics and Clinical Perspectives on AR Gesture Design
  7. Methods
  8. Study Overview
  9. Gesture Corpus
  10. System for Embodied Gesture Design
  11. Participants
  12. Procedure
  13. Data Sources and Analysis
  14. Ethics, Validity, and Scope
  15. Results
  16. ...and 35 more sections

Figures (16)

  • Figure 1: Prototype for gesture elicitation: (a) authoring in Lens Studio; (b) rendering the Rotate cue.
  • Figure 2: PT-informed gesture design code book used in open coding (Round 1--2). Codes consolidate four biomechanical principles and articulate how PTs evaluated gesture sustainability and substitution.
  • Figure 3: PT reasoning around "Throw". Round 1 revealed intuitive overhand and push-shot mimicry; Round 2 replaced these with elbow-anchored, low-amplitude variants that redirect rotation to the forearm and reduce shoulder load.
  • Figure 4: Comparison of unsustainable versus PT-recommended workspace postures. PTs uniformly redirected users from high-elevation, abducted arm positions to a compact chest-level workspace anchored by the elbow.
  • Figure 5: PT substitution logic for Enlarge / Move-a-little: shifting effort from distal-only gestures (single-finger, wrist-led) to proximal anchoring via elbow-driven stabilization, multi-finger load distribution, and tripod-like grips.
  • ...and 11 more figures