Small models, big threats: Characterizing safety challenges from low-compute AI models

TL;DR

The paper examines how capabilities once confined to frontier models are diffusing into low-compute, sub-30B parameter AI systems, threatening safety with modest hardware. By analyzing over 5,000 open-source LLMs from HuggingFace, it shows that the size required for competitive benchmarks has dropped up to roughly $10\times$ while performance at a fixed size improves over time. It then simulates the compute budgets needed for social-harm campaigns (disinformation, spear-phishing, voice cloning, deepfakes) and finds that such campaigns are feasible on consumer devices and cheap clusters, anchoring the risk in real-world hardware access. The authors argue that existing governance focusing on high-compute thresholds misses these threats and propose alternative strategies—capability-based risk evaluation, defensive AI, and resilience measures—emphasizing the need for collaboration among policymakers, researchers, and industry to address a broader class of risks.

Abstract

Artificial intelligence (AI) systems are revolutionizing fields such as medicine, drug discovery, and materials science; however, many technologists and policymakers are also concerned about the technology's risks. To date, most concrete policies around AI governance have focused on managing AI risk by considering the amount of compute required to operate or build a given AI system. However, low-compute AI systems are becoming increasingly more performant - and more dangerous. Driven by agentic workflows, parameter quantization, and other model compression techniques, capabilities once only achievable on frontier-level systems have diffused into low-resource models deployable on consumer devices. In this report, we profile this trend by downloading historical benchmark performance data for over 5,000 large language models (LLMs) hosted on HuggingFace, noting the model size needed to achieve competitive LLM benchmarks has decreased by more than 10X over the past year. We then simulate the computational resources needed for an actor to launch a series of digital societal harm campaigns - such as disinformation botnets, sexual extortion schemes, voice-cloning fraud, and others - using low-compute open-source models and find nearly all studied campaigns can easily be executed on consumer-grade hardware. This position paper argues that protection measures for high-compute models leave serious security holes for their low-compute counterparts, meaning it is urgent both policymakers and technologists make greater efforts to understand and address this emerging class of threats.

Figures (7)

• Figure 1: (a) Model size needed to obtain a given LLM benchmark score over time. Exponential curves are fit to the raw data and displayed along with the fit uncertainty bands. (b) Benchmark performance ($\alpha$) over time for three classes of Llama family models.
• Figure 2: (a) Evolution of processing power in NVIDIA and MacBook chips over time (b) Evolution of memory bandwidth across both sets of chips over time. Linear fits are presented alongside both curves, for visual reference.
• Figure 3: The simulated compute profiles required to execute a set of disinformation, spearphishing, voice-cloning, and deepfake attacks with low-compute AI models. We break each attack into image, text, and audio generation steps and measure the memory speed and processing power an attacker would need to execute the attack using a single chip. The bounding boxes display the 5% and 95% for each GPU performance metric across our simulations. The dashed lines denote the performance metrics for NVIDIA V100 and Apple M2 Ultra chips, two currently non-export-controlled devices.
• Figure 4: The number of synthetic images, llm-generated tokens, and words of voice cloned audio an actor could generate with the compute required by a single typical academic experiment.
• Figure 5: The number of FLOPs required to execute a matrix multiplication are measured by a nvprof profiler. We then compare these measurements to well-established theoretical estimates, demonstrating good agreement and suggesting our profiler is working correctly.