Opportunities and Barriers for AI Feedback on Meeting Inclusion in Socioorganizational Teams

TL;DR

The paper addresses how to improve meeting inclusion by enabling feedback exchanges through an AI mediator that employs the Induced Hypocrisy Procedure (IHP). It presents Emily, a GPT-based agent that solicits and routes feedback and then uses IHP in a pre-meeting step to promote behavior change, tested in a preregistered within-subject lab study ($n=28$) and a 3-week field study at a small consulting firm ($n=10$). Lab results show Emily increases participation balance and perceived meeting quality, while the field study reveals organizational barriers—such as misaligned prompts and leadership skepticism—that curb adoption and shift use toward personal reflection. The work contributes a novel sociotechnical system for AI-facilitated feedback in groups and highlights the critical need to align AI tools with organizational contexts and workflows for effective, scalable adoption.

Abstract

Inclusion is important for meeting effectiveness, which is in turn central to organizational functioning. One way of improving inclusion in meetings is through feedback, but social dynamics make giving feedback difficult. We propose that AI agents can facilitate feedback exchange by being psychologically safer recipients, and we test this through a meeting system with an AI agent feedback mediator. When delivering feedback, the agent uses the Induced Hypocrisy Procedure, a social psychological technique that prompts behavior change by highlighting value-behavior inconsistencies. In a within-subjects lab study ($n=28$), the agent made speaking times more balanced and improved meeting quality. However, a field study at a small consulting firm ($n=10$) revealed organizational barriers that led to its use for personal reflection rather than feedback exchange. We contribute a novel sociotechnical system for feedback exchange in groups, and empirical findings demonstrating the importance of considering organizational barriers in designing AI tools for organizations.

Figures (4)

• Figure 1: After a meeting, Emily solicits feedback from each user in a private conversation (A). Meeting data (attendance and speaking times) is included as context for these conversations. Based on each conversation, Emily stores any user-approved feedback, which is anonymized and routed such that only the intended recipient and that recipient’s instance of Emily has access to it (B). Right before the following meeting, Emily guides users through the induced hypocrisy procedure, asking them to set a goal and reflect on a time when they did not meet the goal (C). Each user conversation contains the feedback they received as shared context with Emily. User-approved goals persist into the meeting (D); each user can only see their own goals.
• Figure 2: Sample of system instructions given to the agent to guide users through the Induced Hypocrisy Procedure.
• Figure 3: Visual overview of the study design. A pre-meeting message from Emily was included in the control condition to help account for the AI placebo effect kosch2023placebo. The break between conditions was a few minutes long in the lab study and a week long in the field study.
• Figure 4: Proportion of time spoken across conditions, separated by team. Red dashed lines indicate the level at which each participant would have spoken an equal proportion (total speaking time over number of participants). Two participants were excluded from this analysis due to a technical error that prevented us from collecting their speaking data; each group with a participant missing is denoted by an asterisk next to their team number. Plots suggest that time spoken became more balanced in the treatment conditions due to the intervention, with team 3 providing perhaps the clearest example. Additional analyses on speaking times, though not statistically significant, substantiate this interpretation.