SHARPIE: A Modular Framework for Reinforcement Learning and Human-AI Interaction Experiments
Hüseyin Aydın, Kevin Godin-Dubois, Libio Goncalvez Braz, Floris den Hengst, Kim Baraka, Mustafa Mert Çelikok, Andreas Sauter, Shihan Wang, Frans A. Oliehoek
TL;DR
The paper addresses the need for a generic, modular platform to study how humans and RL agents interact in diverse, dynamic settings. It introduces SHARPIE, a Python-based, Gymnasium-compatible framework with a versatile environment/algorithm wrapper, a multi-modal web UI, logging, and deployment utilities, enabling experiments across interactive reward specification, human feedback, action delegation, preference elicitation, user modeling, and human–AI teaming. It presents motivating use cases and situates SHARPIE within the landscape of related work, arguing that it provides a standard for human–RL interactions in multi-agent contexts and emphasizes interoperability with existing libraries. The authors outline a design that supports plug-in environments and libraries, with future plans for richer modalities and hosted deployments to broaden accessibility and impact in human–AI collaboration research and education.
Abstract
Reinforcement learning (RL) offers a general approach for modeling and training AI agents, including human-AI interaction scenarios. In this paper, we propose SHARPIE (Shared Human-AI Reinforcement Learning Platform for Interactive Experiments) to address the need for a generic framework to support experiments with RL agents and humans. Its modular design consists of a versatile wrapper for RL environments and algorithm libraries, a participant-facing web interface, logging utilities, deployment on popular cloud and participant recruitment platforms. It empowers researchers to study a wide variety of research questions related to the interaction between humans and RL agents, including those related to interactive reward specification and learning, learning from human feedback, action delegation, preference elicitation, user-modeling, and human-AI teaming. The platform is based on a generic interface for human-RL interactions that aims to standardize the field of study on RL in human contexts.