Conversational Agents to Facilitate Deliberation on Harmful Content in WhatsApp Groups

TL;DR

This study investigates using a conversational agent to facilitate deliberation on harmful content within WhatsApp groups, addressing moderation and fact-checking gaps posed by end-to-end encryption. A design probe with 21 Indian WhatsApp users and semi-structured interviews reveal that anonymity and neutrality are highly valued, but activation, representation, workload, and group dynamics pose design tensions. Findings suggest deliberation can improve critical reflection and diverse perspectives, even if it does not directly replace fact-checking, by providing an intermediate, decentralized pathway for addressing harmful content. The work contributes design recommendations and a theoretical lens for evaluating deliberative systems in private, encrypted group chats, highlighting implications for CSCW and HCI research on scalable, user-centered moderation alternatives.

Abstract

WhatsApp groups have become a hotbed for the propagation of harmful content including misinformation, hate speech, polarizing content, and rumors, especially in Global South countries. Given the platform's end-to-end encryption, moderation responsibilities lie on group admins and members, who rarely contest such content. Another approach is fact-checking, which is unscalable, and can only contest factual content (e.g., misinformation) but not subjective content (e.g., hate speech). Drawing on recent literature, we explore deliberation -- open and inclusive discussion -- as an alternative. We investigate the role of a conversational agent in facilitating deliberation on harmful content in WhatsApp groups. We conducted semi-structured interviews with 21 Indian WhatsApp users, employing a design probe to showcase an example agent. Participants expressed the need for anonymity and recommended AI assistance to reduce the effort required in deliberation. They appreciated the agent's neutrality but pointed out the futility of deliberation in echo chamber groups. Our findings highlight design tensions for such an agent, including privacy versus group dynamics and freedom of speech in private spaces. We discuss the efficacy of deliberation using deliberative theory as a lens, compare deliberation with moderation and fact-checking, and provide design recommendations for future such systems. Ultimately, this work advances CSCW by offering insights into designing deliberative systems for combating harmful content in private group chats on social media.

Figures (4)

• Figure 1: The probe showed a fictional informal WhatsApp group among friends where sometimes harmful content is shared. The agent is shown in Figure \ref{['fig:bot_design']} in the appendix.
• Figure 2: Different phases where the agent interacts with WhatsApp group members.
• Figure 3: After demoing the agent, participants were shown one of these three harmful messages. They were asked the same questions the agent would ask them: to rate the accuracy of the message on a scale of 1--10 and provide reasoning behind their rating. The three messages were sourced from popular fact-checking websites in India.
• Figure 4: Summary of findings from our study