Performance Smells in ML and Non-ML Python Projects: A Comparative Study

TL;DR

This study investigates performance smells in Python by comparing ML and non-ML projects using a large GitHub-based dataset and the RIdiom tool. It presents a dual analysis: (1) prevalence and types of smells across project types, and (2) distribution of smells across the ML pipeline stages using a novel hybrid classifier for stage labeling. The key findings show ML projects exhibit a higher density of performance smells, particularly Assign Multi Targets, Truth Value Test, and Chain Compare, with Data Processing identified as the most affected pipeline stage. The work offers practical, stage-aware recommendations for developers and tooling and highlights directions for researchers to refine detection methods and assess real-world impact on training time and energy efficiency.

Abstract

Python is widely adopted across various domains, especially in Machine Learning (ML) and traditional software projects. Despite its versatility, Python is susceptible to performance smells, i.e., suboptimal coding practices that can reduce application efficiency. This study provides a comparative analysis of performance smells between ML and non-ML projects, aiming to assess the occurrence of these inefficiencies while exploring their distribution across stages in the ML pipeline. For that, we conducted an empirical study analyzing 300 Python-based GitHub projects, distributed across ML and non-ML projects, categorizing performance smells based on the RIdiom tool. Our results indicate that ML projects are more susceptible to performance smells likely due to the computational and data-intensive nature of ML workflows. We also observed that performance smells in the ML pipeline predominantly affect the Data Processing stage. However, their presence in the Model Deployment stage indicates that such smells are not limited to the early stages of the pipeline. Our findings offer actionable insights for developers, emphasizing the importance of targeted optimizations for smells prevalent in ML projects. Furthermore, our study underscores the need to tailor performance optimization strategies to the unique characteristics of ML projects, with particular attention to the pipeline stages most affected by performance smells.

Figures (6)

• Figure 1: Distribution of performance smells per KLOC ML vs Non-ML
• Figure 2: Histogram of smells/KLOC
• Figure 3: Distribution of Truth Value Test per KLOC
• Figure 4: Distribution of Truth Value Test per sick files
• Figure 5: Distribution of Performance Smells in Multi-labelled files Across ML Pipeline Stages (%)