Meeting effectiveness and inclusiveness: large-scale measurement, identification of key features, and prediction in real-world remote meetings

TL;DR

This study tackles the challenge of measuring meeting effectiveness and inclusiveness at organizational scale in real-world remote/hybrid meetings by embedding end-of-meeting surveys within a CMC system across five organizations and linking subjective ratings to rich telemetry. It extends a descriptive graph modeling approach (EIM) to identify generalizable drivers of effective and inclusive meetings and builds a predictive model using telemetry (via LightGBM), while thoroughly examining data quality issues such as survey rating skew and response timing. Key findings show that vocal participation is the strongest driver of inclusiveness, smaller meetings improve outcomes, and video presence enhances participation in small, short meetings, while screen-sharing can boost perceived effectiveness but may dampen participation. However, predictive power for individual ratings from telemetry alone remains limited, highlighting the continued importance of survey data for accurate measurement, and underscoring privacy and deployment considerations for large-scale real-world systems. The work offers concrete design and deployment guidance for organizations seeking scalable, data-driven improvements to meeting culture in the era of distributed work, and outlines promising directions for richer telemetry and privacy-preserving analyses to further enhance predictive capabilities.

Abstract

Workplace meetings are vital to organizational collaboration, yet relatively little progress has been made toward measuring meeting effectiveness and inclusiveness at scale. The recent rise in remote and hybrid meetings represents an opportunity to do so via computer-mediated communication (CMC) systems. Here, we share the results of an effective and inclusive meetings survey embedded within a CMC system in a diverse set of companies and organizations. We correlate the survey results with objective metrics available from the CMC system to identify the generalizable attributes that characterize perceived effectiveness and inclusiveness in meetings. Additionally, we explore a predictive model of meeting effectiveness and inclusiveness based solely on objective meeting attributes. Lastly, we show challenges and discuss solutions around the subjective measurement of meeting experiences. To our knowledge, this is the largest data-driven study conducted after the pandemic peak to measure, understand, and predict effectiveness and inclusiveness in real-world meetings at an organizational scale.

Figures (9)

• Figure 1: Initial End-of-Meeting Survey. We dropped the second page and only showed the first page according to randomized A/B experiment results.
• Figure 2: The graphical model showing the network of conditional dependencies between meeting attributes and Effective and Inclusive. The red and green edges show negative and positive dependencies, respectively. The weight on each edge is the adjusted Odds Ratio as a comparable measure of strength for each dependency.
• Figure 3: Model predictions for the impact of Participation in the probability of Inclusive experience for different Inclusive rate baselines. The impact declines for participants who would have had a very high Inclusive rate (over $60\%$, already very good experience) or a very low Inclusive rate (lower than $35\%$, significantly poor experience) without participating in conversations.
• Figure 4: Participation rate drops as Meeting Size increases. The rapid decline begins with more than 5 people in the meeting. The difference in Participation is largest and statistically significant when we compare 8-or-less-participant meetings with more-than-8-participant meetings.
• Figure 5: EIM metrics by media. The least Effective and Inclusive experiences are audio-only calls. Video-only calls are most Inclusive ($2\%$ higher than audio-only), but not the most Effective. $95\%$ Confidence intervals are shown for each metric to enable comparisons.