Breathless: An 8-hour Performance Contrasting Human and Robot Expressiveness

TL;DR

The robot technology behind an original performance that pairs a human dancer (Cuan) with an industrial robot arm for an eight-hour dance that unfolds over the timespan of an American workday is described.

Abstract

This paper describes the robot technology behind an original performance that pairs a human dancer (Cuan) with an industrial robot arm for an eight-hour dance that unfolds over the timespan of an American workday. To control the robot arm, we combine a range of sinusoidal motions with varying amplitude, frequency and offset at each joint to evoke human motions common in physical labor such as stirring, digging, and stacking. More motions were developed using deep learning techniques for video-based human-pose tracking and extraction. We combine these pre-recorded motions with improvised robot motions created live by putting the robot into teach-mode and triggering force sensing from the robot joints onstage. All motions are combined with commercial and original music using a custom suite of python software with AppleScript, Keynote, and Zoom to facilitate on-stage communication with the dancer. The resulting performance contrasts the expressivity of the human body with the precision of robot machinery. Video, code and data are available on the project website: https://sites.google.com/playing.studio/breathless

Figures (9)

• Figure 1: Catie Cuan with UR5e industrial robot arm at the National Sawdust Theater, Dec 16, 2023.
• Figure 2: Dancer Catie Cuan with the output of the OpenPose software, indicating automatically segmented kinematic joints. We process these images to extract joint angles over time, then fit these trajectories to sinusoidal functions and use these to program the robot arm.
• Figure 3: Three snapshots from phase two, where the human dancer returns to the lab to dance with the UR5e robot arm. The graphs below each image were rendered using matplotlib to illustrate the joint angles extracted from OpenPose.
• Figure 4: Visualization of the UR5e Arm. 6 joints on the UR5e utilized by the urdfpy simulation, a Python library for loading, manipulating, and exporting URDF files and robot specifications. Figure 4b showcases the UR5e with shoulder$\_$lift$\_$joint at an angle of -$\pi$/4 radians and elbow$\_$joint at $\pi$/4 radians. Figure 4c illustrates the UR5e with shoulder$\_$lift$\_$joint at an angle of -$\pi$/4 radians and elbow$\_$joint at an angle of 0 radians.
• Figure 5: Methodology Behind Stirring Action Development Example of pipeline for setting sinusoidal parameter values for robot motions that suggest the robot arm is stirring a pot of soup.