VR-Assisted Guide Dog Training: A 360° PanoHaptic System for Right-Hand Commands Analysis

TL;DR

This work tackles the shortage of qualified guide-dog trainers by proposing a VR pano-haptic system that lets novices re-experience training from an experienced trainer’s perspective and practice right-hand commands with real-time visual and tactile cues. It integrates 360° video capturing, pose estimation, and data-driven analysis of dog status and command poses, plus four auxiliary modes to tailor practice and evaluation. The study records four diverse training datasets, analyzes head and arm poses to derive command timing and dog responses, and translates these insights into actionable visual and haptic feedback within a VR environment. The approach has the potential to accelerate skill acquisition, improve training quality, and increase guide-dog availability, addressing a critical mobility aid gap, with planned future work on practicality evaluations and interactive dog models.

Abstract

This paper presents a VR-based guide dog training system designed to assist novice trainers in understanding guide dog behavior and issuing appropriate training commands. Guide dogs play a vital role in supporting independent mobility for visually impaired individuals, yet the limited number of skilled trainers restricts their availability. Training is highly demanding, requiring accurate observation of the dog's status and precise command issuance, especially through right-hand gestures. While the trainer's left hand holds the harness to perceive haptic cues, the right hand is used to indicate directions, maintain attention, and provide comfort, with motion patterns varying by scenario and the dog's progress. Currently, novices learn mainly by observing experts or watching videos, which lacks immersion and makes it difficult to adopt the trainer's perspective for understanding behavior or synchronizing command timing. To address these limitations, the proposed system introduces a VR-based assistive platform integrating panoramic visuals and haptic feedback to create an immersive training environment. The visual module provides contextual guidance, including cues for command execution and real-time comparison of the user's posture with standard actions, while the haptic module delivers tactile feedback for command gestures. Users can re-experience training sessions across diverse scenarios and dog proficiency levels, allowing independent and repeated practice. By improving the timing, accuracy, and expressiveness of right-hand commands, the system aims to accelerate skill acquisition, enhance training quality, and mitigate the shortage of qualified trainers, ultimately increasing the availability of guide dogs for visually impaired individuals.

Figures (8)

• Figure 1: Recording Setup: 360° camera mounted on the left shoulder of the trainer along training tracks including training room, inside the building and outdoors
• Figure 2: Labeling for Key Points Training and Recognition: Three sets of key points are labeled separately for objects being the arms(yellow dots), the guide dog(white dots) and the marker(red dots).
• Figure 3: Illustration Figures of Calculation for (a) Dog Postures (b)Command Poses
• Figure 4: Distribution of Head Pose towards Trainer when Commands are Issued across Datasets: Distribution of Head Poses mainly falls into three categories, with examples of corresponding scenarios:(1) the dog in waiting condition but dog body not tilted; (2)the dog in waiting condition with dog body tilting towards the trainer; (3)during walking conditions.
• Figure 5: Distribution of Yaw Angles of Instruction Commands across Datasets: Distribution of yaw angles of instruction commands shows the commands mainly occupy three poses: (1) Rightward Commands for Attracting Attention; (2) Lateral-direction: Commands for Controlling and Directional Commands for turning; (3) Directional Commands indicating left-front directions.