Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PuppetAI: A Customizable Platform for Designing Tactile-Rich Affective Robot Interaction

Jiaye Li, Tongshun Chen, Siyi Ma, Elizabeth Churchill, Ke Wu

TL;DR

PuppetAI addresses the need for expressive, tactile-friendly human-robot interaction by providing a modular soft-robot platform that combines a scalable cable-driven continuum gesture framework with a four-layer perceptual-affective software stack. An affective expression loop translates vocal input into parameterized gestures, leveraging an LLM-driven motion library to produce real-time social feedback. Key contributions include a hardware design with deformable and rigid segments, a gesture library for discrete and continuous actions, and a software pipeline that integrates voice processing, emotion analysis, and motion scheduling. This approach lowers production costs and increases customization, enabling broader adoption in tactile-based expressive robotics and enabling researchers to explore multimodal affective interactions across diverse puppet morphologies.

Abstract

We introduce PuppetAI, a modular soft robot interaction platform. This platform offers a scalable cable-driven actuation system and a customizable, puppet-inspired robot gesture framework, supporting a multitude of interaction gesture robot design formats. The platform comprises a four-layer decoupled software architecture that includes perceptual processing, affective modeling, motion scheduling, and low-level actuation. We also implemented an affective expression loop that connects human input to the robot platform by producing real-time emotional gestural responses to human vocal input. For our own designs, we have worked with nuanced gestures enacted by "soft robots" with enhanced dexterity and "pleasant-to-touch" plush exteriors. By reducing operational complexity and production costs while enhancing customizability, our work creates an adaptable and accessible foundation for future tactile-based expressive robot research. Our goal is to provide a platform that allows researchers to independently construct or refine highly specific gestures and movements performed by social robots.

PuppetAI: A Customizable Platform for Designing Tactile-Rich Affective Robot Interaction

TL;DR

PuppetAI addresses the need for expressive, tactile-friendly human-robot interaction by providing a modular soft-robot platform that combines a scalable cable-driven continuum gesture framework with a four-layer perceptual-affective software stack. An affective expression loop translates vocal input into parameterized gestures, leveraging an LLM-driven motion library to produce real-time social feedback. Key contributions include a hardware design with deformable and rigid segments, a gesture library for discrete and continuous actions, and a software pipeline that integrates voice processing, emotion analysis, and motion scheduling. This approach lowers production costs and increases customization, enabling broader adoption in tactile-based expressive robotics and enabling researchers to explore multimodal affective interactions across diverse puppet morphologies.

Abstract

We introduce PuppetAI, a modular soft robot interaction platform. This platform offers a scalable cable-driven actuation system and a customizable, puppet-inspired robot gesture framework, supporting a multitude of interaction gesture robot design formats. The platform comprises a four-layer decoupled software architecture that includes perceptual processing, affective modeling, motion scheduling, and low-level actuation. We also implemented an affective expression loop that connects human input to the robot platform by producing real-time emotional gestural responses to human vocal input. For our own designs, we have worked with nuanced gestures enacted by "soft robots" with enhanced dexterity and "pleasant-to-touch" plush exteriors. By reducing operational complexity and production costs while enhancing customizability, our work creates an adaptable and accessible foundation for future tactile-based expressive robot research. Our goal is to provide a platform that allows researchers to independently construct or refine highly specific gestures and movements performed by social robots.
Paper Structure (11 sections, 1 figure, 1 table)

This paper contains 11 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Robotic Puppetry
  4. Cable-driven continuum robots
  5. Design of the PuppetAI
  6. Cable-driven Continuum Robot Gesture Framework
  7. Software Control Architecture
  8. Interactive Loop
  9. Expressive Gestures Design
  10. Interaction Scenarios
  11. Conclusion

Figures (1)

  • Figure 1: Overview of the continuum robot framework design. The robot’s structure consists of deformable and non-deformable sections, with the bending range adjusted by dividing the flexible part into specific units. It is powered by base-mounted motors and cables, using elastic ropes to provide a restoring force and simplify the control of planar motions.