Interactive Multi-Robot Flocking with Gesture Responsiveness and Musical Accompaniment

TL;DR

This work introduces interactive multi-robot flocking that integrates gesture responsiveness and musical accompaniment to create engaging human-robot experiences in everyday environments. It combines a Boids-based base controller with four subsystems (Head, Arm, Base, Music Mode), gesture-driven responses, and seven weight modes learned through supervised data to produce varied, expressive flocking dynamics. A model-prediction and choreographer-guided experiment evaluates participant perception and gesture engagement, revealing significant differences in Anthropomorphism and Likeability across conditions and providing qualitative insight into the engaging potential of expressive robot movement. The study demonstrates how learning-based control and choreographic design can enrich social robotics, enabling scalable, aesthetic interactions that blend navigation, gesture, and music for public-facing robotic performances.

Abstract

For decades, robotics researchers have pursued various tasks for multi-robot systems, from cooperative manipulation to search and rescue. These tasks are multi-robot extensions of classical robotic tasks and often optimized on dimensions such as speed or efficiency. As robots transition from commercial and research settings into everyday environments, social task aims such as engagement or entertainment become increasingly relevant. This work presents a compelling multi-robot task, in which the main aim is to enthrall and interest. In this task, the goal is for a human to be drawn to move alongside and participate in a dynamic, expressive robot flock. Towards this aim, the research team created algorithms for robot movements and engaging interaction modes such as gestures and sound. The contributions are as follows: (1) a novel group navigation algorithm involving human and robot agents, (2) a gesture responsive algorithm for real-time, human-robot flocking interaction, (3) a weight mode characterization system for modifying flocking behavior, and (4) a method of encoding a choreographer's preferences inside a dynamic, adaptive, learned system. An experiment was performed to understand individual human behavior while interacting with the flock under three conditions: weight modes selected by a human choreographer, a learned model, or subset list. Results from the experiment showed that the perception of the experience was not influenced by the weight mode selection. This work elucidates how differing task aims such as engagement manifest in multi-robot system design and execution, and broadens the domain of multi-robot tasks.

Figures (11)

• Figure 1: Autonomously flocking robots move nearby a participating human in this interactive experience.
• Figure 2: The overall flocking system. Individual robots connect to the parent client running on a workstation. This parent client creates individual flocking service objects for each robot, which includes 4 subservices to manage different behaviors.
• Figure 3: The system components for an individual robot. The four main components are the Base, Arm, Head, and Music Mode.
• Figure 4: The robots' light rings turn yellow when a human is detected in the scene. The head controller directs the robot to look at the closest human in the boundary area if one is detected.
• Figure 5: Three gestures lead to different robot actions. Left: the "Hands Together" action, robots gaze upwards and the light rings turn orange. Center: the "Left Hand Up" action, robots open and close their grippers and the light rings turn dark blue. Right: the "Right Hand Up" action, robots turn in place and the light rings turn green.