Robot-Wearable Conversation Hand-off for Navigation

TL;DR

The paper addresses cognitive load in indoor navigation by introducing a conversation hand-off that transfers a CA from a stationary robot to a wearable. The system uses a local server architecture with Mimic 3 for TTS, Whisper for ASR, and RASA for dialogue, evaluated in a within-subject study with $N=24$ participants across Robot-Only, Wearable-Only, and Hand-off conditions on a university campus. Results show wearable embodiment is preferred, the hand-off is engaging, and there are no significant navigation-performance gains, with mean task times of $146.4$ s and average response times of $2.89$ s (robot), $4.26$ s (watch), and $3.96$ s (hand-off). Guided by these findings, the paper yields design considerations for shared voice/state, explicit triggers, and using the robot as an attention anchor to support cognitive augmentation through multi-embodiment public AI assistants.

Abstract

Navigating large and complex indoor environments, such as universities, airports, and hospitals, can be cognitively demanding and requires attention and effort. While mobile applications provide convenient navigation support, they occupy the user's hands and visual attention, limiting natural interaction. In this paper, we explore conversation hand-off as a method for multi-device indoor navigation, where a Conversational Agent (CA) transitions seamlessly from a stationary social robot to a wearable device. We evaluated robot-only, wearable-only, and robot-to-wearable hand-off in a university campus setting using a within-subjects design with N=24 participants. We find that conversation hand-off is experienced as engaging, even though no performance benefits were observed, and most preferred using the wearable-only system. Our findings suggest that the design of such re-embodied assistants should maintain a shared voice and state across embodiments. We demonstrate how conversational hand-offs can bridge cognitive and physical transitions, enriching human interaction with embodied AI.

Figures (7)

• Figure 1: Usage of the conversation hand-off system presented as a storyboard, where the user looks for a cafe.
• Figure 2: Photographs showing the user interface of the wearable navigation assistant embodiment. Holding the button while speaking (left), and when the processing is finished, the assistant is ready for another input (right).
• Figure 3: Conversation hand-off GUI on the Robot's chest tablet.
• Figure 4: The participant uses the robot situated in front of the booth to begin navigation (left) and uses the watch as she makes her way through the landmarks (middle) and fills out questionnaires at the start point before, between and after the three conditions right).
• Figure 5: Box plot comparing NASA-RTLX scores for conditions.