Modeling Communication Perception in Development Teams Using Monte Carlo Methods

TL;DR

This study investigates how perception of communication sentiment varies within development teams and how many members must report sentiment to accurately reflect team mood. Using a Monte Carlo approach on data from 45 developers labeling 100 SA4SE statements, it derives a preliminary model for the minimum agreement needed from a subset given the whole-team agreement, demonstrating that omitting even one member in a 7-person team can materially distort mood estimates. The work provides a regression-based lower-bound formula $f(n,k,\hat{\kappa})$ for $\,\min \kappa_n$ and shows that the agreement volatility grows rapidly as the subset size shrinks, emphasizing the practical need to include all team members in mood surveys. It also discusses ethical considerations, limitations, and avenues for future work, including framework development and longitudinal studies to better integrate mood assessments into software engineering practice.

Abstract

Software development is a collaborative task involving diverse development teams, where toxic communication can negatively impact team mood and project success. Mood surveys enable the early detection of underlying tensions or dissatisfaction within development teams, allowing communication issues to be addressed before they escalate, fostering a positive and productive work environment. The mood can be surveyed indirectly by analyzing the text-based communication of the team. However, emotional subjectivity leads to varying sentiment interpretations across team members; a statement perceived neutrally by one developer might be seen as problematic by another developer with a different conversational culture. Early identification of perception volatility can help prevent misunderstandings and enhance team morale while safeguarding the project. This paper analyzes the diversity of perceptions within arbitrary development teams and determines how many team members should report their sentiment to accurately reflect the team's mood. Through a Monte Carlo experiment involving 45 developers, we present a preliminary mathematical model to calculate the minimum agreement among a subset of developers based on the whole team's agreement. This model can guide leadership in mood assessment, demonstrating that omitting even a single member in an average-sized 7-member team can misrepresent the overall mood. Therefore, including all developers in mood surveying is recommended to ensure a reliable evaluation of the team's mood.

Figures (2)

• Figure 1: Violin plot of the agreement values $\boldsymbol{\kappa_n}$ resulting from one run of the Monte Carlo experiment (cf. Table \ref{['table:mc']}). Each violin and their inner box plot contain $\boldsymbol{m = 1000}$ agreement values $\boldsymbol{\kappa_n}$ corresponding to a subset of $\boldsymbol{n}$ out of $\boldsymbol{k = 45}$ participants.
• Figure 2: Comparison of the minimum agreement $\boldsymbol{\min \kappa_n}$ from the Monte Carlo experiment and our deducted model $\boldsymbol{f(n, k, \hat{\kappa})}$ used for regression.