PeaTMOSS: A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software

TL;DR

PeaTMOSS tackles the lack of a comprehensive, queryable PTM ecosystem dataset. It builds a large-scale repository linking 281,638 PTMs with 28,575 downstream GitHub projects and 44,337 PTM–GitHub mappings, enriched by LLM-extracted metadata. The paper reports first supply-chain statistics and a license-compatibility analysis, highlighting metadata gaps and rare license conflicts. The dataset enables systematic study of PTM development, reuse patterns, and downstream impact, and supports tooling for model search, comparison, and licensing awareness.

Abstract

The development and training of deep learning models have become increasingly costly and complex. Consequently, software engineers are adopting pre-trained models (PTMs) for their downstream applications. The dynamics of the PTM supply chain remain largely unexplored, signaling a clear need for structured datasets that document not only the metadata but also the subsequent applications of these models. Without such data, the MSR community cannot comprehensively understand the impact of PTM adoption and reuse. This paper presents the PeaTMOSS dataset, which comprises metadata for 281,638 PTMs and detailed snapshots for all PTMs with over 50 monthly downloads (14,296 PTMs), along with 28,575 open-source software repositories from GitHub that utilize these models. Additionally, the dataset includes 44,337 mappings from 15,129 downstream GitHub repositories to the 2,530 PTMs they use. To enhance the dataset's comprehensiveness, we developed prompts for a large language model to automatically extract model metadata, including the model's training datasets, parameters, and evaluation metrics. Our analysis of this dataset provides the first summary statistics for the PTM supply chain, showing the trend of PTM development and common shortcomings of PTM package documentation. Our example application reveals inconsistencies in software licenses across PTMs and their dependent projects. PeaTMOSS lays the foundation for future research, offering rich opportunities to investigate the PTM supply chain. We outline mining opportunities on PTMs, their downstream usage, and cross-cutting questions.

Figures (12)

• Figure 1: This paper presents the PeaTMOSS dataset: Pre-Trained Models in Open- Source Software. PeaTMOSS includes data on 281,638 pre-trained models, 28,575 GitHub repositories that use pre-trained models, and 44,337 links between them.
• Figure 2: The PTM supply chain. Engineers publish PTM packages to model registries. PTMs are used by applications and other PTMs.
• Figure 3: PeaTMOSS data schema. There are four regions: tables for PTMs (basic \ref{['sec:original-Peatmoss']} and enhanced \ref{['sec:enhanced-peatmoss']}), tables for GitHub projects, and a table of PTM-Application dependency relations. Tables link to PTM and GitHub snapshots in a Globus share. Our artifact has a navigable version (\ref{['sec:DataAvailability']}).
• Figure 4: Example use of two HuggingFace PTMs. The code initializes a tokenizer (AutoTokenizer) and a model (AutoModelForMaskedLM) from the transformers library for a multilingual BERT model.
• Figure 5: Number of projects that access PTMs from each Hugging Face library, as captured via Sourcegraph search. Note: Log scale.