Exploring The Impact Of Proactive Generative AI Agent Roles In Time-Sensitive Collaborative Problem-Solving Tasks

TL;DR

Design considerations for proactive generative AI agents are provided based on findings on two forms of proactive support in digital escape rooms: a facilitator agent that offered summaries and group structures, and a peer agent that proposed ideas and answered queries.

Abstract

Collaborative problem-solving under time pressure is common but difficult, as teams must generate ideas quickly, coordinate actions, and track progress. Generative AI offers new opportunities to assist, but we know little about how proactive agents affect the dynamics of real-time, co-located teamwork. We studied two forms of proactive support in digital escape rooms: a facilitator agent that offered summaries and group structures, and a peer agent that proposed ideas and answered queries. In a within-subjects study with 24 participants, we compared group performance and processes across three conditions: no AI, peer, and facilitator. Results show that the peer agent occasionally enhanced problem-solving by offering timely hints and memory support; however, it also disrupted flow, increased workload, and created over-reliance. In comparison, the facilitator agent provided light scaffolding but had a limited impact on outcomes. We provide design considerations for proactive generative AI agents based on our findings.

Figures (11)

• Figure 1: Diagram of Screen 2, Puzzle 1 with the facilitator agent condition. The gray boxes represent the puzzle elements. There are two main features of the facilitator agent (black box at the top): (A) The green text field shows where Fiona suggested structured collaboration strategies like the 1-2-4-All liberating structure mccandless_liberating_2020, provided time reminders, and asked groups to divide up unsolved parts of the puzzle; (B) The blue text field displays Fiona's summary of ideas discussed by group, presented every three minutes.
• Figure 2: Diagram of Screen 1, Puzzle 1, with the peer agent condition. The gray boxes represent the puzzle elements. There are three main features of the peer agent (black box on the right): (A) Ava proactively shared brainstorming thoughts every 3 minutes (displayed in the green text field), based on puzzle screenshots and contextualized by group conversations; (B) The blue text field indicates that groups could follow up by asking Ava to explain or vary its ideas; and (C) Ava was available as a chat-based partner on each puzzle screen, responding to user queries.
• Figure 3: Timeline of proactive interventions from the facilitator (top row) and peer (bottom) agents during the 20-minute session
• Figure 4: Themes across the participants that describe their experiences when solving a puzzle with generative AI during the formative study (FS). Participant quotes are marked as FS-P<id> in the themes.
• Figure 5: Recording of Group 6, Session 2 (Puzzle 2 with the peer agent). The top panels, (a) and (b), capture the two puzzle screens, each displayed on separate TVs connected to laptops. (c) Each participant wore a lavalier microphone, and their audio was merged into a single channel and transcribed in real time using WhisperX bain2022whisperx. These transcripts were used by the peer agent to contextualize responses and by the facilitator to generate summaries. (d) Finally, a room camera captured group interactions and overall activity throughout the session.