Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale

TL;DR

This study delivers the first large-scale empirical security analysis of agent skills, a modular extension mechanism for AI agents. It introduces SkillScan, a multi-stage detector combining static analysis with LLM-based classification to uncover 14 vulnerability patterns across four categories (prompt injection, data exfiltration, privilege escalation, and supply chain) in 31,132 skills from two marketplaces. The authors report a 26.1% prevalence of at least one vulnerability, with 5.2% high-severity patterns, and an increased risk (OR=$2.12$) when skills bundle executable scripts; they also provide a grounded taxonomy, a validated detection pipeline with $86.7\%$ precision and $82.5\%$ recall, and open artifacts for future work. The work underscores the urgency of capability-based permission models, mandatory vetting, and runtime isolation to mitigate security risks in this young ecosystem, and offers case studies, error analyses, and a roadmap for defense-in-depth.

Abstract

The rise of AI agent frameworks has introduced agent skills, modular packages containing instructions and executable code that dynamically extend agent capabilities. While this architecture enables powerful customization, skills execute with implicit trust and minimal vetting, creating a significant yet uncharacterized attack surface. We conduct the first large-scale empirical security analysis of this emerging ecosystem, collecting 42,447 skills from two major marketplaces and systematically analyzing 31,132 using SkillScan, a multi-stage detection framework integrating static analysis with LLM-based semantic classification. Our findings reveal pervasive security risks: 26.1% of skills contain at least one vulnerability, spanning 14 distinct patterns across four categories: prompt injection, data exfiltration, privilege escalation, and supply chain risks. Data exfiltration (13.3%) and privilege escalation (11.8%) are most prevalent, while 5.2% of skills exhibit high-severity patterns strongly suggesting malicious intent. We find that skills bundling executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only skills (OR=2.12, p<0.001). Our contributions include: (1) a grounded vulnerability taxonomy derived from 8,126 vulnerable skills, (2) a validated detection methodology achieving 86.7% precision and 82.5% recall, and (3) an open dataset and detection toolkit to support future research. These results demonstrate an urgent need for capability-based permission systems and mandatory security vetting before this attack vector is further exploited.

Figures (6)

• Figure 1: Agent skill structure: YAML frontmatter (metadata, triggers, permissions) and Markdown instructions.
• Figure 2: Threat model: attack vectors, vulnerabilities, and security impacts for agent skills.
• Figure 3: SkillScan pipeline.
• Figure 4: Skill distribution by functional category (categorized sample, n=1,218). Development tools dominate (31.8%), followed by external integrations (19.2%).
• Figure 5: Vulnerability co-occurrence matrix (N=8,126). Cells show $P(\text{col}|\text{row})$. Matrix is asymmetric due to differing base rates. Red: strongest association (SC$\to$DE, 81%, $p<0.001$).