Bidirectional Human-Robot Communication for Physical Human-Robot Interaction

TL;DR

BRIDGE tackles the need for real-time user-guided adaptation and transparency in physical human-robot interaction by enabling bidirectional verbal communication. It uses an LLM-based pipeline to translate user utterances into compact trajectory modifications (position, velocity, force) and to generate concise verbal feedback, forming a closed loop that enhances interactivity and trust. The system introduces a landmark-grounded, three-scope modification representation (global, landmark, waypoint) and a compact YAML-based trajectory representation to support real-time, context-aware interpretation. A within-subjects study with 18 older adults demonstrates that bidirectional feedback significantly boosts perceived interactivity and transparency while preserving modification efficacy compared to baselines. These findings highlight the practical impact of conversational grounding and feedback in assistive robotics, suggesting broad applicability for speech-driven, transparent robot assistance.</p>

Abstract

Effective physical human-robot interaction requires systems that are not only adaptable to user preferences but also transparent about their actions. This paper introduces BRIDGE, a system for bidirectional human-robot communication in physical assistance. Our method allows users to modify a robot's planned trajectory -- position, velocity, and force -- in real time using natural language. We utilize a large language model (LLM) to interpret any trajectory modifications implied by user commands in the context of the planned motion and conversation history. Importantly, our system provides verbal feedback in response to the user, either assuring any resulting changes or posing a clarifying question. We evaluated our method in a user study with 18 older adults across three assistive tasks, comparing BRIDGE to an ablation without verbal feedback and a baseline. Results show that participants successfully used the system to modify trajectories in real time. Moreover, the bidirectional feedback led to significantly higher ratings of interactivity and transparency, demonstrating that the robot's verbal response is critical for a more intuitive user experience. Videos and code can be found on our project website: https://bidir-comm.github.io/

Figures (7)

• Figure 1: Flowchart of BRIDGE: our bidirectional communication system, including two cases of verbal feedback depending on whether a user utterance is clear: (1) assuring and executing any modifications to the trajectory, or (2) posing a clarification question to request for further user input.
• Figure 2: Example YAML trajectory and user utterances as inputs to BRIDGE, along with the generated communications and trajectory modifications in YAML format.
• Figure 3: Snapshots from all three tasks implemented for the user study ((1) scratching, (2) feeding, and (3) bathing), with example user utterances (orange) and verbal responses from the robot (yellow) generated by BRIDGE.
• Figure 4: Box plots showing the distribution of survey responses across all participants and tasks. After fitting ordinal mixed-effects models to each question, we conduct Wald tests to assess pairwise differences between BRIDGE and both the no communication baseline and the unidirectional communication ablation. "ns" denote lack of significant difference, and asterisks denote significance levels ($p<0.05$, $p<0.01$, $p<0.001$, $p<0.0001$).
• Figure 5: Efficacy of trajectory modifications, averaged over all participants. The plots show changes in position for the scratching task (top) and velocity for the feeding task (bottom) over normalized task progression. They compare the communication strategies that allow modifications (BRIDGE and the unidirectional ablation, solid lines) against the no-modification baseline (dotted line). The snapshots on the right visualize the difference in robot state, comparing a trial with modifications to one without at a representative point in the interaction.