"I Felt Bad After We Ignored Her": Understanding How Interface-Driven Social Prominence Shapes Group Discussions with GenAI

TL;DR

This paper presents a GenAI-based conversational agent that can actively engage in spoken dialogue during video calls and designs three distinct collaboration modes that vary the social prominence of the agent by manipulating its presence in the shared space and the degree of control users have over its participation.

Abstract

Recent advancements in the conversational and social capabilities of generative AI (GenAI) have sparked interest in its role as an agent capable of actively participating in human-AI group discussions. Despite this momentum, we don't fully understand how GenAI shapes conversational dynamics or how the interface design impacts its influence on the group. In this paper, we introduce interface-driven social prominence as a design lens for collaborative GenAI systems. We then present a GenAI-based conversational agent that can actively engage in spoken dialogue during video calls and design three distinct collaboration modes that vary the social prominence of the agent by manipulating its presence in the shared space and the degree of control users have over its participation. A mixed-methods within-subjects study, in which 18 dyads engaged in realistic discussions with a GenAI agent, offers empirical insights into how communication patterns and the collective negotiation of GenAI's influence shift based on how it is embedded into the collaborative experience. Based on these findings, we outline design implications for supporting the coordination and critical engagement required in human-AI groups.

Figures (9)

• Figure 1: Mapping of collaboration modes by interface-driven social prominence of GenAI agent. Roundtable provides high agent presence in the shared space with low user control, peripheral provides medium presence with medium control, and Breakout provides low presence within the human-AI group with high control.
• Figure 2: Conceptual overview of the underlying system and decision tree used to steer the mechanisms in our GenAI agent's response and turn-taking during a real-time discussion over video chat.
• Figure 3: Our system supported a variety of interactions between Discussants and the GenAI Agent. (A) Discussants can individually enter a breakout room with the agent, with the option to return to the main room at any time. (B) When one discussant is in the breakout room with the agent, their teammate can call them back to the main room. (C) Agent raising its hand within Roundtable mode. (D) Agent raising its hand from the periphery in Peripheral mode.
• Figure 4: An abridged version of the study scenario with descriptions of the three discussion tasks and the assigned preferences for discussant 1 (D1), discussant 2 (D2), and the GenAI agent (A). A full description including worldviews is in appendix \ref{['appendix:study']}.
• Figure 5: The communication patterns of all discussions aggregated across modes and visualized using ONA graphs. The nodes correspond to the codes or type of turns and the node size corresponds its frequency of occurrence. The thickness of the edges connecting the nodes reflect the relative frequency of co-occurrence between two codes, and the arrows on these edges reflect the directed temporal connections between them. Left: Differences in user vs agent turns within overall communication patterns, Right: Users' communication patterns when isolating turns that were either directed to the agent or referencing it.