A Rusty Link in the AI Supply Chain: Detecting Evil Configurations in Model Repositories
Ziqi Ding, Qian Fu, Junchen Ding, Gelei Deng, Yi Liu, Yuekang Li
TL;DR
Addresses security gaps in AI supply chains by focusing on malicious configuration files in Hugging Face repositories. Proposes ConfigScan, a hybrid LLM-based tool that analyzes configuration files together with their runtime code and Readme content, complemented by a rule-based filter. Demonstrates, via large-scale data, that thousands of repositories contain suspicious configurations and that ConfigScan can identify latent threats—including two newly discovered malicious configurations—with lower false positives than rule-based methods. Argues that robust validation of configuration files is essential for secure AI hosting platforms.
Abstract
Recent advancements in large language models (LLMs) have spurred the development of diverse AI applications from code generation and video editing to text generation; however, AI supply chains such as Hugging Face, which host pretrained models and their associated configuration files contributed by the public, face significant security challenges; in particular, configuration files originally intended to set up models by specifying parameters and initial settings can be exploited to execute unauthorized code, yet research has largely overlooked their security compared to that of the models themselves; in this work, we present the first comprehensive study of malicious configurations on Hugging Face, identifying three attack scenarios (file, website, and repository operations) that expose inherent risks; to address these threats, we introduce CONFIGSCAN, an LLM-based tool that analyzes configuration files in the context of their associated runtime code and critical libraries, effectively detecting suspicious elements with low false positive rates and high accuracy; our extensive evaluation uncovers thousands of suspicious repositories and configuration files, underscoring the urgent need for enhanced security validation in AI model hosting platforms.