Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inferring Operator Emotions from a Motion-Controlled Robotic Arm

Xinyu Qi, Zeyu Deng, Shaun Alexander Macdonald, Liying Li, Chen Wang, Muhammad Ali Imran, Philip G. Zhao

TL;DR

This work addresses how an operator's emotions influence remote robot motion and demonstrates that a motion-controlled robotic avatar can inherit and reveal these affective states. By developing DTW and CNN-based classifiers and a feature-rich end-effector/joint motion pipeline, the authors achieve up to 83.3$\\%$ emotion recognition accuracy on a large 6000-instance dataset, outperforming ECG-based baselines in ecological scenarios. The approach enables emotion-aware telerobotics and introduces the concept of emotive-motion dampening to balance safety and operator autonomy. The study provides a foundation for emotionally intelligent HRI in teleoperation, with implications for safety-critical tasks, privacy considerations, and future multimodal extensions.

Abstract

A remote robot operator's affective state can significantly impact the resulting robot's motions leading to unexpected consequences, even when the user follows protocol and performs permitted tasks. The recognition of a user operator's affective states in remote robot control scenarios is, however, underexplored. Current emotion recognition methods rely on reading the user's vital signs or body language, but the devices and user participation these measures require would add limitations to remote robot control. We demonstrate that the functional movements of a remote-controlled robotic avatar, which was not designed for emotional expression, can be used to infer the emotional state of the human operator via a machine-learning system. Specifically, our system achieved 83.3$\%$ accuracy in recognizing the user's emotional state expressed by robot movements, as a result of their hand motions. We discuss the implications of this system on prominent current and future remote robot operation and affective robotic contexts.

Inferring Operator Emotions from a Motion-Controlled Robotic Arm

TL;DR

This work addresses how an operator's emotions influence remote robot motion and demonstrates that a motion-controlled robotic avatar can inherit and reveal these affective states. By developing DTW and CNN-based classifiers and a feature-rich end-effector/joint motion pipeline, the authors achieve up to 83.3 emotion recognition accuracy on a large 6000-instance dataset, outperforming ECG-based baselines in ecological scenarios. The approach enables emotion-aware telerobotics and introduces the concept of emotive-motion dampening to balance safety and operator autonomy. The study provides a foundation for emotionally intelligent HRI in teleoperation, with implications for safety-critical tasks, privacy considerations, and future multimodal extensions.

Abstract

A remote robot operator's affective state can significantly impact the resulting robot's motions leading to unexpected consequences, even when the user follows protocol and performs permitted tasks. The recognition of a user operator's affective states in remote robot control scenarios is, however, underexplored. Current emotion recognition methods rely on reading the user's vital signs or body language, but the devices and user participation these measures require would add limitations to remote robot control. We demonstrate that the functional movements of a remote-controlled robotic avatar, which was not designed for emotional expression, can be used to infer the emotional state of the human operator via a machine-learning system. Specifically, our system achieved 83.3 accuracy in recognizing the user's emotional state expressed by robot movements, as a result of their hand motions. We discuss the implications of this system on prominent current and future remote robot operation and affective robotic contexts.

Paper Structure

This paper contains 47 sections, 12 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Modeling Emotion
  4. Emotion Recognition during Human-Computer and Human-Robot Interaction
  5. Emotion Recognition Using Human Individual Status Data
  6. Physiological Signals
  7. Behavioral Signals
  8. Robot Emotion Expression
  9. System Overview & Feasibility Pre-Study
  10. Motion-controlled Robotic Avatar Platform
  11. Pre-Study: Feasibility of Emotion Classification on Robotic Avatar
  12. System Architecture
  13. Emotion Classification Algorithm Design
  14. Dimensional model of emotions
  15. Data Segmentation, Normalization, and Calibration
  16. ...and 32 more sections

Figures (12)

  • Figure 1: Our robot platform with motion-based emotion transmission.
  • Figure 2: The emotions of a human impacting the trajectory of the motion-controlled robot arm they operate. We investigate how the robot's movement can be used to infer these operator emotions.
  • Figure 3: Robot's 3D movement trajectory, velocity, acceleration and jerk plot for joy, annoyance and neutral emotions.
  • Figure 4: The architecture of the proposed human emotion inference through the robotic avatar.
  • Figure 5: The four emotional states we induced in this work, Joy, Pleasure, Annoyance, and Sadness mapped to each of the four quadrants of Russel's circumplex model of emotion, with neutral at the origin.
  • ...and 7 more figures