The RoSiD Tool: Empowering Users to Design Multimodal Signals for Human-Robot Collaboration

TL;DR

The paper addresses the challenge of diverse human-robot communication preferences by introducing RoSiD, a tool that lets users self-specify multimodal signals (visual, auditory, kinetic) for collaborative robots using query-based and search-based interfaces. It combines a preference-learning framework with multimodal embeddings from CLIP, VGGish, and a GRU Seq2Seq model, and a clustering-based method to generate signals from user data. The study reports good usability (SUS median 75) and learning effects, with evidence that prior user data improves initial signal alignment across modalities. The work highlights RoSiD's potential to personalize robot communication in real-world tasks and maps out future work to extend the approach to other embodiments and compare personalized versus generic signaling strategies.

Abstract

Robots that cooperate with humans must be effective at communicating with them. However, people have varied preferences for communication based on many contextual factors, such as culture, environment, and past experience. To communicate effectively, robots must take those factors into consideration. In this work, we present the Robot Signal Design (RoSiD) tool to empower people to easily self-specify communicative preferences for collaborative robots. We show through a participatory design study that the RoSiD tool enables users to create signals that align with their communicative preferences, and we illuminate how this tool can be further improved.

Figures (4)

• Figure 1: Interfaces for the RoSiD tool.
• Figure 2: Structure of the design session with approximate times for each section.
• Figure 3: Box plots showing the times users spent deigning signals.
• Figure 4: Box plots comparing the alignment of initial queries based on random suggestions and the proposed clustered suggestions.