Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

React to This! How Humans Challenge Interactive Agents using Nonverbal Behaviors

Chuxuan Zhang, Bermet Burkanova, Lawrence H. Kim, Lauren Yip, Ugo Cupcic, Stéphane Lallée, Angelica Lim

TL;DR

A human behavior codebook of 188 unique nonverbal behaviors used by humans to test the virtual characters to help build more interaction-aware and believable robots, especially when humans push them to their limits.

Abstract

How do people use their faces and bodies to test the interactive abilities of a robot? Making lively, believable agents is often seen as a goal for robots and virtual agents but believability can easily break down. In this Wizard-of-Oz (WoZ) study, we observed 1169 nonverbal interactions between 20 participants and 6 types of agents. We collected the nonverbal behaviors participants used to challenge the characters physically, emotionally, and socially. The participants interacted freely with humanoid and non-humanoid forms: a robot, a human, a penguin, a pufferfish, a banana, and a toilet. We present a human behavior codebook of 188 unique nonverbal behaviors used by humans to test the virtual characters. The insights and design strategies drawn from video observations aim to help build more interaction-aware and believable robots, especially when humans push them to their limits.

React to This! How Humans Challenge Interactive Agents using Nonverbal Behaviors

TL;DR

A human behavior codebook of 188 unique nonverbal behaviors used by humans to test the virtual characters to help build more interaction-aware and believable robots, especially when humans push them to their limits.

Abstract

How do people use their faces and bodies to test the interactive abilities of a robot? Making lively, believable agents is often seen as a goal for robots and virtual agents but believability can easily break down. In this Wizard-of-Oz (WoZ) study, we observed 1169 nonverbal interactions between 20 participants and 6 types of agents. We collected the nonverbal behaviors participants used to challenge the characters physically, emotionally, and socially. The participants interacted freely with humanoid and non-humanoid forms: a robot, a human, a penguin, a pufferfish, a banana, and a toilet. We present a human behavior codebook of 188 unique nonverbal behaviors used by humans to test the virtual characters. The insights and design strategies drawn from video observations aim to help build more interaction-aware and believable robots, especially when humans push them to their limits.
Paper Structure (16 sections, 4 figures, 5 tables)

This paper contains 16 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Research Questions
  5. Setup and Materials
  6. Pilot Study
  7. Study Design
  8. Data Collection and Analysis
  9. Segmentation
  10. Segment classification
  11. Additional labeling
  12. Thematic Analysis
  13. Results and Discussion
  14. Limitations and Future Work
  15. Conclusion
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Physical setup of our study: participant interacting with virtual character (left), and teleoperator using face and upper body tracking to control the virtual character (right).
  • Figure 2: Participants were asked to test 6 interactive virtual characters physically, emotionally, and socially.
  • Figure 3: (LEFT) Illustrative examples for physical, emotional, and social nonverbal human behaviors. Physical behaviors included changing posture, proxemic movement, and mock physical contact. (RIGHT) Examples of character-specific interactions related to physical affordance (banana peeling, toilet flushing) role play (fish swimming, robot dancing), and the agent's contextual environment and accessories (shivering from cold, pulling on shirt).
  • Figure 4: Compound Behavior Examples: behaviors consisting of sequential actions (top) and behaviors consisting of multimodal actions (bottom).