Use of Winsome Robots for Understanding Human Feedback (UWU)

TL;DR

This work investigates how perceived cuteness in robots influences human evaluative feedback during learning from human feedback, a key factor in reinforcement learning from human input. The authors conduct a within-subject pilot using MOVO/Jaco robots with two face designs (cute vs control) to measure perceived cuteness and feedback biases, and they extend TAMER with a stochastic mechanism that flips suboptimal positive feedback to negative with probability $p$ when the user bias is detected. Results show a significant difference in perceived cuteness and a trend toward higher positive feedback for the cute robot, with implications for learning dynamics in HRI. The proposed stochastic TAMER framework aims to mitigate positive-feedback bias and improve learning efficiency, offering a practical pathway to more robust human-guided robot learning in realistic, aesthetic-influenced interactions.

Abstract

As social robots become more common, many have adopted cute aesthetics aiming to enhance user comfort and acceptance. However, the effect of this aesthetic choice on human feedback in reinforcement learning scenarios remains unclear. Previous research has shown that humans tend to give more positive than negative feedback, which can cause failure to reach optimal robot behavior. We hypothesize that this positive bias may be exacerbated by the robot's level of perceived cuteness. To investigate, we conducted a user study where participants critique a robot's trajectories while it performs a task. We then analyzed the impact of the robot's aesthetic cuteness on the type of participant feedback. Our results suggest that there is a shift in the ratio of positive to negative feedback when perceived cuteness changes. In light of this, we experiment with a stochastic version of TAMER which adapts based on the user's level of positive feedback bias to mitigate these effects.

Figures (8)

• Figure 1: This figure depicts a robotic pick-and-place task. We record positive and negative feedback provided by users on the robot's task execution. Our paper explores the impact of robotic aesthetics, such as perceived cuteness, on users’ feedback tendencies and subsequent robot learning outcomes.
• Figure 2: Illustration of the robot’s task. The MOVO must pick up the plushie from the starting position (current placement) and place it into the goal.
• Figure 3: Robots faces used in the study. Control on the left and Cute on the right. The cute robot has a face with the appearance of larger eyes and forehead, features derived from the baby schema b11.
• Figure 4: Comparison of the distribution of positive and negative feedback for the two robots. Users were less likely to give negative feedback for the cute than the control (left), and overall gave more positive feedback to the cute robot than the control (right).
• Figure 5: Wumpus World with robot agent (top left). Obstacles include holes and a monster. Goal state is the treasure box (bottom right).