Analyzing the Evolution and Maintenance of ML Models on Hugging Face

TL;DR

This study analyzes the evolution and maintenance of ML models on the Hugging Face Hub by mining data for over 380,000 models using the HF Hub API and the HFCommunity dataset. It combines time-series analyses, topic modeling (LDA), author-network clustering (Louvain), and commit classification (DistilBERT) to characterize community dynamics and maintenance practices, including a two-class maintenance framework (high vs low). Key findings show exponential growth in HF activity, dominance of Transformers and PyTorch, concentrated author groups driving most popularity, NSFW content trends in model cards, and a maintenance pattern dominated by perfective commits with substantial variation across models. The work offers practical guidance for selecting well-maintained models, proposes a maintenance-focused framework and replication package, and highlights tool and process gaps for ML-specific maintenance on community platforms.

Abstract

Hugging Face (HF) has established itself as a crucial platform for the development and sharing of machine learning (ML) models. This repository mining study, which delves into more than 380,000 models using data gathered via the HF Hub API, aims to explore the community engagement, evolution, and maintenance around models hosted on HF, aspects that have yet to be comprehensively explored in the literature. We first examine the overall growth and popularity of HF, uncovering trends in ML domains, framework usage, authors grouping and the evolution of tags and datasets used. Through text analysis of model card descriptions, we also seek to identify prevalent themes and insights within the developer community. Our investigation further extends to the maintenance aspects of models, where we evaluate the maintenance status of ML models, classify commit messages into various categories (corrective, perfective, and adaptive), analyze the evolution across development stages of commits metrics and introduce a new classification system that estimates the maintenance status of models based on multiple attributes. This study aims to provide valuable insights about ML model maintenance and evolution that could inform future model development strategies on platforms like HF.

Figures (8)

• Figure 1: Popularity metrics evolution on HF.
• Figure 2: Evolution of the relative popularity
• Figure 4: Time series analysis of model cards LDA topics
• Figure 5: # of commits and commit size per model histograms
• Figure 6: Average number of commits for each model quarterly