Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration

TL;DR

The paper tackles how to study human–LLM-agent collaboration as genuine partnerships rather than mere tools. It introduces an open, configurable research platform with four modules (participant interface, researcher interface, ACP, and experiment controller) and a declarative Experiment Configuration Language (ECL) to enable reproducible manipulation of theory-grounded interaction controls. A central feature is the Agent Context Protocol (ACP) that enforces human–agent parity in perception and action spaces, allowing controlled comparisons to classic CSCW experiments. Through two case studies (Shape Factory and Hidden Profile) and a cognitive walkthrough with HCI researchers, the work demonstrates that the platform can faithfully re-implement traditional experiments, reveal how interaction designs shape collaboration, and support iterative usability improvements. Overall, the platform provides a methodological foundation for systematic, evidence-based exploration of human–agent collaboration and open-science sharing of configurations and results.

Abstract

Intelligent systems have traditionally been designed as tools rather than collaborators, often lacking critical characteristics that collaboration partnerships require. Recent advances in large language model (LLM) agents open new opportunities for human-LLM-agent collaboration by enabling natural communication and various social and cognitive behaviors. Yet it remains unclear whether principles of computer-mediated collaboration established in HCI and CSCW persist, change, or fail when humans collaborate with LLM agents. To support systematic investigations of these questions, we introduce an open and configurable research platform for HCI researchers. The platform's modular design allows seamless adaptation of classic CSCW experiments and manipulation of theory-grounded interaction controls. We demonstrate the platform's research efficacy and usability through three case studies: (1) two Shape Factory experiments for resource negotiation with 16 participants, (2) one Hidden Profile experiment for information pooling with 16 participants, and (3) a participatory cognitive walkthrough with five HCI researchers to refine workflows of researcher interface for experiment setup and analysis.

Figures (20)

• Figure 1: Existing research-focused platforms for collaboration studies. From left to right, the top row: Share greenberg1990sharing, GroupKit roseman1996building, LabintheWild reinecke2015labinthewild, Empirica almaatouq2021empirica; the bottom row: oTree chen2016otree, nodeGame balietti2017nodegame, Overcooked carroll2019utility, Pairit ju2025collaborating.
• Figure 2: An example of the participant interface for the Shape Factory experiment bos2004group. The interface consists of five fully configurable modules: My Status displays the participant's private status, My Actions displays permitted actions, My Tasks displays the tasks to be completed, Social provides available interaction channels, and Information Dashboard shows other participants' real-time information. The experiment's timer is shown at the top.
• Figure 3: The workflow of HCI researchers for study setup through the researcher interface, which includes five steps: experiment selection, parameter configuration, interaction control, participant registration, and participant customization.
• Figure 4: The agent context protocol (ACP) provides a standardized channel between the experiment controller and LLM agents, regardless of agent architecture designs. ACP includes four modules: Experiment Rules (task setup and constraints), Experiment Specifications (participant roles and resources), Permitted Actions with Schemas (structured formats for communication and trades), and Private and Public States (information visible to participants and agents).
• Figure 5: Bar charts of collaboration outcomes in the $CS_{CL}$ case study, showing (left to right) average wealth, successful trades, message volume, and human participants’ wealth across session order.