An Empirical Study on Remote Code Execution in Machine Learning Model Hosting Ecosystems

TL;DR

The paper provides the first large-scale cross-platform empirical study of remote code execution risks during model loading in five major model hubs, revealing that about 2–5% of repositories require potentially dangerous custom loading code, amounting to over 46,000 affected artifacts. Through multi-tool static analysis (Bandit, CodeQL, Semgrep) and signature-based YARA scans, it uncovers prevalent weaknesses (notably CWE-502, CWE-95, CWE-78) and systemic exposure to injection and deserialization vulnerabilities, with platform-specific risk patterns. Platform mitigation varies widely, ranging from Hugging Face’s automated malware and pickle scanning to minimal protections on other hubs, leading to misalignment between security mechanisms and developer practices. Developer discussions further reveal confusion, risk perception gaps, and a demand for safer, native integration of custom loading code, highlighting the need for enforceable trust boundaries and integrated, cryptographically-backed safety controls. Together, these findings inform platform operators, framework developers, and researchers about tightening security while preserving usability in AI model-sharing infrastructures.

Abstract

Model-sharing platforms, such as Hugging Face, ModelScope, and OpenCSG, have become central to modern machine learning development, enabling developers to share, load, and fine-tune pre-trained models with minimal effort. However, the flexibility of these ecosystems introduces a critical security concern: the execution of untrusted code during model loading (i.e., via trust_remote_code or trust_repo). In this work, we conduct the first large-scale empirical study of custom model loading practices across five major model-sharing platforms to assess their prevalence, associated risks, and developer perceptions. We first quantify the frequency with which models require custom code to function and identify those that execute arbitrary Python files during loading. We then apply three complementary static analysis tools: Bandit, CodeQL, and Semgrep, to detect security smells and potential vulnerabilities, categorizing our findings by CWE identifiers to provide a standardized risk taxonomy. We also use YARA to identify malicious patterns and payload signatures. In parallel, we systematically analyze the documentation, API design, and safety mechanisms of each platform to understand their mitigation strategies and enforcement levels. Finally, we conduct a qualitative analysis of over 600 developer discussions from GitHub, Hugging Face, and PyTorch Hub forums, as well as Stack Overflow, to capture community concerns and misconceptions regarding security and usability. Our findings reveal widespread reliance on unsafe defaults, uneven security enforcement across platforms, and persistent confusion among developers about the implications of executing remote code. We conclude with actionable recommendations for designing safer model-sharing infrastructures and striking a balance between usability and security in future AI ecosystems.

Figures (6)

• Figure 1: Example of a custom configuration script (left) and loading a custom model (right).
• Figure 2: Threat model overview
• Figure 3: Methodology Overview
• Figure 4: Monthly repository creations for custom models.
• Figure 5: Tag distributions of the custom models across different platforms.