No Risk, No Reward: Towards An Automated Measure of Psychological Safety from Online Communication

TL;DR

The paper addresses measuring Psychological Safety (PS) in virtual teams by leveraging data from Slack messages. It derives four PS-related communication categories (voice, supportive, learning, familiarity) and tests them with a mixed-methods approach on two hybrid design teams with contrasting PS levels. The main findings show that interaction signals like the number of replies, emoji reactions, and user mentions distinguish higher-PS teams, while simple keyword frequency does not, highlighting nuanced language use. This work provides a first step toward automated PS monitoring on enterprise communication platforms, with potential for real-time dashboards and timely managerial interventions, while acknowledging limitations and the need for broader validation.

Abstract

The data created from virtual communication platforms presents the opportunity to explore automated measures for monitoring team performance. In this work, we explore one important characteristic of successful teams - Psychological Safety - or the belief that a team is safe for interpersonal risk-taking. To move towards an automated measure of this phenomenon, we derive virtual communication characteristics and message keywords related to elements of Psychological Safety from the literature. Using a mixed methods approach, we investigate whether these characteristics are present in the Slack messages from two design teams - one high in Psychological Safety, and one low. We find that some usage characteristics, such as replies, reactions, and user mentions, might be promising metrics to indicate higher levels of Psychological Safety, while simple keyword searches may not be nuanced enough. We present the first step towards the automated detection of this important, yet complex, team characteristic.

Figures (1)

• Figure 1: Average perceptions of Psychological Safety values per each team over the semester.