How do Machine Learning Models Change?

TL;DR

The paper investigates how open-source ML artifacts on Hugging Face evolve over time by applying an ML-specific change taxonomy to over 680,000 commits from 100,000 models and 2,251 releases, using Bayesian networks to reveal temporal dependencies in commit and release sequences. It finds a shift toward Output Data, Project Metadata, and Sharing in commits and a dominance of Output Data, Sharing, and External Documentation in releases, with distinct lifecycle phases where foundational setup, iterative refinement, and dissemination co-occur. The study also shows that collaboration intensity and model popularity shape evolutionary pathways, with popular projects prioritizing dissemination and infrastructure-focused releases while collaborative teams emphasize documentation. These findings offer concrete guidance for ML maintenance and MLOps, highlighting the need to treat data artifacts, configurations, and documentation as first-class, versioned assets and to design workflows that separate nightly experimentation from formal releases for reliable production use.

Abstract

The proliferation of Machine Learning (ML) models and their open-source implementations has transformed Artificial Intelligence research and applications. Platforms like Hugging Face (HF) enable this evolving ecosystem, yet a large-scale longitudinal study of how these models change is lacking. This study addresses this gap by analyzing over 680,000 commits from 100,000 models and 2,251 releases from 202 of these models on HF using repository mining and longitudinal methods. We apply an extended ML change taxonomy to classify commits and use Bayesian networks to model temporal patterns in commit and release activities. Our findings show that commit activities align with established data science methodologies, such as the Cross-Industry Standard Process for Data Mining (CRISP-DM), emphasizing iterative refinement. Release patterns tend to consolidate significant updates, particularly in model outputs, sharing, and documentation, distinguishing them from granular commits. Furthermore, projects with higher popularity exhibit distinct evolutionary paths, often starting from a more mature baseline with fewer foundational commits in their public history. In contrast, those with intensive collaboration show unique documentation and technical evolution patterns. These insights enhance the understanding of model changes on community platforms and provide valuable guidance for best practices in model maintenance.

Figures (31)

• Figure 1: Examples of commits and releases in HF model repositories.
• Figure 2: Data collection and analysis process
• Figure 3: Dataset Construction Process
• Figure 4: Process of refining the LLM prompt through consecutive evaluation rounds and final validation.
• Figure 5: Evolution of file co-editing clusters across lifecycle stages (based on Louvain community detection on file co-occurrence graph within commits). Stage 1 (left) shows distinct clusters for core model/config files and tokenizer files. Stage 5 (right) shows different groupings, with README.md strongly linked to the model file.