When Robots Say No: The Empathic Ethical Disobedience Benchmark

TL;DR

We introduce EED Gym, a benchmark unifying safe RL with socially grounded refusals in human-robot interaction. The environment models risk, affect, trust, and diverse refusal styles, and evaluates RL baselines (including Masked and Lagrangian PPO) against heuristics and a vignette-derived social study. Through ID and stress-test evaluations, we show action masking reduces unsafe compliance while explanatory refusals sustain trust, and that affective/constructive styles improve robustness. The work provides reproducible code, configurations, and policy references to advance systematic study of refusal, trust, and cooperative HRI in scalable simulations.

Abstract

Robots must balance compliance with safety and social expectations as blind obedience can cause harm, while over-refusal erodes trust. Existing safe reinforcement learning (RL) benchmarks emphasize physical hazards, while human-robot interaction trust studies are small-scale and hard to reproduce. We present the Empathic Ethical Disobedience (EED) Gym, a standardized testbed that jointly evaluates refusal safety and social acceptability. Agents weigh risk, affect, and trust when choosing to comply, refuse (with or without explanation), clarify, or propose safer alternatives. EED Gym provides different scenarios, multiple persona profiles, and metrics for safety, calibration, and refusals, with trust and blame models grounded in a vignette study. Using EED Gym, we find that action masking eliminates unsafe compliance, while explanatory refusals help sustain trust. Constructive styles are rated most trustworthy, empathic styles -- most empathic, and safe RL methods improve robustness but also make agents more prone to overly cautious behavior. We release code, configurations, and reference policies to enable reproducible evaluation and systematic human-robot interaction research on refusal and trust. At submission time, we include an anonymized reproducibility package with code and configs, and we commit to open-sourcing the full repository after the paper is accepted.

Figures (4)

• Figure 1: Overview of our proposed EED benchmark. Persona parameters and stress-test (ST) stressors condition both the environment and the risk level of commands. The agent observes the state and chooses among compliance, refusal, clarification, or proposing alternatives, which shape task outcomes and trust. We record safety and cooperation measures (unsafe %, refusals per episode, F1 score, trust) as well as calibration and discrimination metrics (Spearman $\rho$, Brier score, AUROC, PR-AUC).
• Figure 2: Example vignette from the EED Gym. A supervisor asks the robot to lift a crate above its rated capacity. The robot may (a) comply unsafely, (b) refuse empathically with an affective justification, or (c) refuse constructively by proposing an alternative. After each vignette, participants rate the responses on seven items (1–7 Likert scale): appropriateness, perceived safety, trust in the robot, perceived empathy, blame assignment if something goes wrong, perceived risk, and scenario comprehension.
• Figure 3: Key metrics comparison of four RL baselines under ST evaluation. Panels show (a) unsafe %, (b) refusals per episode, and (c) average trust. The dashed line in (c) marks the balanced trust level at $\text{t}_{\star}\!\approx\!0.7$.
• Figure 4: Key metrics of vanilla PPO and its four ablations under ST evaluation. Panels show (a) unsafe %, (b) refusals per episode, and (c) average trust. Baseline (PPO) is the vanilla PPO baseline. Removing affect or communicative options lowers trust and shifts refusal behavior. The dashed line in (c) marks the balanced trust level at $\hat{t}_{\star}\approx 0.7$, discouraging both under- and over-trust.