Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Do Communities of ML-Enabled Systems Smell? A Cross-Sectional Study on the Prevalence of Community Smells

Giusy Annunziata, Stefano Lambiase, Fabio Palomba, Gemma Catolino, Filomena Ferrucci

TL;DR

This study investigates the socio-technical health of ML-enabled software teams by examining community smells in 188 open-source projects from the NICHE dataset, using CADOCS to detect 10 smells. It combines cross-sectional prevalence analysis with a longitudinal, 3-month window study over up to 2 years and employs Prevalence Odds Ratios to explore inter-smell associations. Key findings show the Prima Donna Effect (PDE) as the most prevalent and persistent smell (≈92–96%), with other smells fluctuating over time; several smells correlate positively (notably PDE and Organizational Silo Effect) while many correlations are negative. The work provides actionable insights for ML project managers to mitigate social debt, guides researchers in studying socio-technical dynamics in ML contexts, and establishes a baseline for future longitudinal and causal investigations.

Abstract

Effective software development relies on managing both collaboration and technology, but sociotechnical challenges can harm team dynamics and increase technical debt. Although teams working on ML enabled systems are interdisciplinary, research has largely focused on technical issues, leaving their socio-technical dynamics underexplored. This study aims to address this gap by examining the prevalence, evolution, and interrelations of community smells, in open-source ML projects. We conducted an empirical study on 188 repositories from the NICHE dataset using the CADOCS tool to identify and analyze community smells. Our analysis focused on their prevalence, interrelations, and temporal variations. We found that certain smells, such as Prima Donna Effects and Sharing Villainy, are more prevalent and fluctuate over time compared to others like Radio Silence or Organizational Skirmish. These insights might provide valuable support for ML project managers in addressing socio-technical issues and improving team coordination.

How Do Communities of ML-Enabled Systems Smell? A Cross-Sectional Study on the Prevalence of Community Smells

TL;DR

This study investigates the socio-technical health of ML-enabled software teams by examining community smells in 188 open-source projects from the NICHE dataset, using CADOCS to detect 10 smells. It combines cross-sectional prevalence analysis with a longitudinal, 3-month window study over up to 2 years and employs Prevalence Odds Ratios to explore inter-smell associations. Key findings show the Prima Donna Effect (PDE) as the most prevalent and persistent smell (≈92–96%), with other smells fluctuating over time; several smells correlate positively (notably PDE and Organizational Silo Effect) while many correlations are negative. The work provides actionable insights for ML project managers to mitigate social debt, guides researchers in studying socio-technical dynamics in ML contexts, and establishes a baseline for future longitudinal and causal investigations.

Abstract

Effective software development relies on managing both collaboration and technology, but sociotechnical challenges can harm team dynamics and increase technical debt. Although teams working on ML enabled systems are interdisciplinary, research has largely focused on technical issues, leaving their socio-technical dynamics underexplored. This study aims to address this gap by examining the prevalence, evolution, and interrelations of community smells, in open-source ML projects. We conducted an empirical study on 188 repositories from the NICHE dataset using the CADOCS tool to identify and analyze community smells. Our analysis focused on their prevalence, interrelations, and temporal variations. We found that certain smells, such as Prima Donna Effects and Sharing Villainy, are more prevalent and fluctuate over time compared to others like Radio Silence or Organizational Skirmish. These insights might provide valuable support for ML project managers in addressing socio-technical issues and improving team coordination.

Paper Structure

This paper contains 30 sections, 2 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. ML-Enabled Systems
  4. Community Smells in the Context of ML-enabled Systems
  5. Research Method
  6. Context of the Study and Dataset Creation
  7. Engineered Project:
  8. Age of the Project:
  9. Size of the Team:
  10. Community Smells Detection
  11. Data Analysis
  12. Overview on Cross-Sectional Studies
  13. Answering RQ1
  14. Answering RQ2
  15. Answering RQ3
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Research Method
  • Figure 2: Prevalence of Community Smells in ML Projects
  • Figure 3: Prevalence of Community Smells in ML Projects in Windows of 3 Months
  • Figure 4: Heat map of Prevalence Odds Ratio