Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Painted Heart Beats

Angshu Adhya, Cindy Yang, Emily Wu, Rishad Hasan, Abhishek Narula, Patrícia Alves-Oliveira

TL;DR

Painted Heart Beats presents AURA, a biometrics-informed human-robot painting framework that augments collaboration between artists and a robotic painter. By integrating EmotiBit heart-rate data, verbal commands, real-time pose estimation, and image-generation and speech-to-text modules within a ROS 2/MoveIt 2 pipeline, the system adapts robot actions and proximity to the artist. The work demonstrates heartbeat-driven proximity control and multimodal interaction, offering insights into how biometric signals can guide creative robot behavior and suggesting extensions with additional biometrics and natural-language interfaces. The contributions establish a blueprint for fluid, adaptive, emotion-aware artistry in human-robot collaborations with potential implications for creative practices and assistive robotic systems.

Abstract

In this work we present AURA, a framework for synergistic human-artist painting. We developed a robot arm that collaboratively paints with a human artist. The robot has an awareness of the artist's heartbeat through the EmotiBit sensor, which provides the arousal levels of the painter. Given the heartbeat detected, the robot decides to increase proximity to the artist's workspace or retract. If a higher heartbeat is detected, which is associated with increased arousal in human artists, the robot will move away from that area of the canvas. If the artist's heart rate is detected as neutral, indicating the human artist's baseline state, the robot will continue its painting actions across the entire canvas. We also demonstrate and propose alternative robot-artist interactions using natural language and physical touch. This work combines the biometrics of a human artist to inform fluent artistic interactions.

Painted Heart Beats

TL;DR

Painted Heart Beats presents AURA, a biometrics-informed human-robot painting framework that augments collaboration between artists and a robotic painter. By integrating EmotiBit heart-rate data, verbal commands, real-time pose estimation, and image-generation and speech-to-text modules within a ROS 2/MoveIt 2 pipeline, the system adapts robot actions and proximity to the artist. The work demonstrates heartbeat-driven proximity control and multimodal interaction, offering insights into how biometric signals can guide creative robot behavior and suggesting extensions with additional biometrics and natural-language interfaces. The contributions establish a blueprint for fluid, adaptive, emotion-aware artistry in human-robot collaborations with potential implications for creative practices and assistive robotic systems.

Abstract

In this work we present AURA, a framework for synergistic human-artist painting. We developed a robot arm that collaboratively paints with a human artist. The robot has an awareness of the artist's heartbeat through the EmotiBit sensor, which provides the arousal levels of the painter. Given the heartbeat detected, the robot decides to increase proximity to the artist's workspace or retract. If a higher heartbeat is detected, which is associated with increased arousal in human artists, the robot will move away from that area of the canvas. If the artist's heart rate is detected as neutral, indicating the human artist's baseline state, the robot will continue its painting actions across the entire canvas. We also demonstrate and propose alternative robot-artist interactions using natural language and physical touch. This work combines the biometrics of a human artist to inform fluent artistic interactions.

Paper Structure

This paper contains 13 sections, 2 figures.

Table of Contents

  1. Introduction and Related Work
  2. Wearables within Arts
  3. Robots that Paint
  4. AURA System
  5. Artist-Robot Interaction
  6. Painting Together Using Verbal Interactions
  7. Heartbeat and Proximity
  8. Artist Impressions
  9. Future Work
  10. Physically Moving the Robot In and Out of the Canvas
  11. Additional Biometrics
  12. Voice Integration
  13. System Questions

Figures (2)

  • Figure 1: Robot reaction to increased heart rate. When the robot detects increased user heart rate (as shown in the graph), the robot disengages from the artist's active workspace temporarily to reduce user heart rate.
  • Figure 2: AURA System Setup. In addition to base CoFRIDA components, like the InstructPix2Pix image generation module, new inputs include user biometric data (heart rate), verbal commands, and additional camera views, allowing for seamless artist-robot interaction during the co-painting process.