Augmenting team diversity and performance by enabling agency and fairness criteria in recommendation algorithms

TL;DR

The results show that participants assigned by an algorithm to work in highly diverse teams struggled to work with different and unfamiliar individuals, while participants enabled by an algorithm to choose collaborators without fairness criteria formed homogenous teams without the necessary skills.

Abstract

In this study, we examined the impact of recommendation systems' algorithms on individuals' collaborator choices when forming teams. Different algorithmic designs can lead individuals to select one collaborator over another, thereby shaping their teams' composition, dynamics, and performance. To test this hypothesis, we conducted a 2 x 2 between-subject laboratory experiment with 332 participants who assembled teams using a recommendation system. We tested four algorithms that controlled the participants' agency to choose collaborators and the inclusion of fairness criteria. Our results show that participants assigned by an algorithm to work in highly diverse teams struggled to work with different and unfamiliar individuals, while participants enabled by an algorithm to choose collaborators without fairness criteria formed homogenous teams without the necessary skills. In contrast, combining users' agency and fairness criteria in an algorithm enhanced teams' performance and composition. This study breaks new ground by providing insights into how algorithms can augment team formation.

Figures (7)

• Figure 1: Study outline. This diagram describes how the research team conducted the data collection process.
• Figure 2: Results of team composition per condition: (a) Surface-level diversity score is the sum of gender, age, race, and ethnicity diversity scores, (b) the deep-level diversity score is the sum of the six project skills diversity scores assessed in the survey, (c) the total diversity score is the sum of the surface-level and deep-level scores, (d) one of the most significant differences across the conditions was gender. Brackets represent statistically significant differences between two conditions using Tukey HSD tests ($p_{adj} < .05$). Each bracket shows its respective p-adjusted value.
• Figure 3: Participants' perceptions of the recommendation system: (a) Participants in the self-assembled and fairness-aware teams were more satisfied with their collaborator choices compared to the algorithmically-diverse teams. The difference between self-assembled teams and random teams was also statistically significant. (b) Participants in self-assembled teams perceived higher quality of the recommendations than participants in the random teams and algorithmically-diverse teams. (c) Participants in the agentic conditions considered the recommendation system more useful to reflect on their collaborators than participants in the algorithmically-diverse teams. (d) No statistically significant differences were found in usability across all conditions. Brackets represent statistically significant differences between two conditions using Tukey HSD tests ($p_{adj} < .05$). Each bracket shows its respective p-adjusted value.
• Figure 4: Team performance scores per condition: Average score per team considering recruitment materials' originality and uniqueness. The difference between conditions (3) and (4) is statistically significant (Tukey HSD test, $p_{adj} < .05$). Each bracket shows its respective p-adjusted value. Number of observations: 83 teams.
• Figure 5: Mediation Model: Only arrows with p-values < .10 are displayed. The interaction term represents the condition with both agency and diversity criteria incorporated in the algorithm. Total observations: 83 teams. $\chi^2: 19.576$, Degrees of freedom = 4, ($p < .001$). Standardized Root Mean Square Residual (SRMR): .04. Comparative Fit Index (CFI): .879. RMSEA = .223. Stars indicate the statistical significance of each coefficient: * $p < .05$, ** $p < .01$, *** $p < .001$.