Impact of Different Failures on a Robot's Perceived Reliability

TL;DR

Which robot failures are in higher need of repair in a human-robot interaction, and how trust could be recovered by robot successes are highlighted.

Abstract

Robots fail, potentially leading to a loss in the robot's perceived reliability (PR), a measure correlated with trustworthiness. In this study we examine how various kinds of failures affect the PR of the robot differently, and how this measure recovers without explicit social repair actions by the robot. In a preregistered and controlled online video study, participants were asked to predict a robot's success in a pick-and-place task. We examined manipulation failures (slips), freezing (lapses), and three types of incorrect picked objects or place goals (mistakes). Participants were shown one of 11 videos -- one of five types of failure, one of five types of failure followed by a successful execution in the same video, or a successful execution video. This was followed by two additional successful execution videos. Participants bet money either on the robot or on a coin toss after each video. People's betting patterns along with a qualitative analysis of their survey responses highlight that mistakes are less damaging to PR than slips or lapses, and some mistakes are even perceived as successes. We also see that successes immediately following a failure have the same effect on PR as successes without a preceding failure. Finally, we show that successful executions recover PR after a failure. Our findings highlight which robot failures are in higher need of repair in a human-robot interaction, and how trust could be recovered by robot successes.

Figures (6)

• Figure 1: 6 of the 11 conditions examined in this study. From left to right: a) Success. The robot puts the bottle in the trash. b) Slip. The robot is unable to pick up the bottle. c) Lapse. The robot positions to pick up the bottle, then freezes for 15 seconds. d) Mistake (Thermos). The robot picks up the reusable thermos and places it in the trash. e) Mistake (Stapler). The robot picks up the stapler and places it into the trash. f) Mistake (Marker). The robot picks up the marker and places it into the mug. Not shown: The other five conditions, which are the failure modes (b--f) immediately followed by success.
• Figure 2: The experimental setup. The robot (blue) attempted to place the disposable plastic water bottle (pink) into the trash (green). In some videos, the robot interacted with other objects (dotted red). Participants saw an unlabeled version of this before placing bets.
• Figure 3: The flow of the experiment. Participants were asked to place bets on robot success before and after seeing three videos---one of the experimental conditions, then two success videos.
• Figure 4: Change in participant bet after they see the condition video, compared to the bet before. Each light dot represents one participant, the dark dot is the mean of participants, and the whiskers are the 95% confidence interval of the mean. We see evidence for differences in effect of failure conditions (slip, mistake, and lapse), with no evidence for difference in effect if the demonstration contains success. Mistake and Success are abbreviated as M and S, respectively.
• Figure 5: Participant betting behavior before any videos (bet 1), after the Slip video (bet 2), and after additional success videos (bets 3 and 4). Perceived reliability (PR) drops after seeing the robot slip. PR rises after seeing success, ending above the baseline from Bet 1.