Spill The Beans: Exploiting CPU Cache Side-Channels to Leak Tokens from Large Language Models

TL;DR

Spill The Beans demonstrates a hardware-level cache side-channel attack that leaks LLM tokens by monitoring embedding-layer cache activity on co-located CPU/GPU servers. By co-locating attacker and victim processes and exploiting unified memory coherency, the method recovers tokens in real time using Flush+Reload on embedding vectors stored in CPU/GPU shared memory. The study shows that attackers can recover 40% of plain English tokens in single-shot scenarios and up to 80–90% of high-entropy credentials like API keys, with leakage improving through strategic token monitoring and repeated prompts. The work highlights a new class of privacy risks in multi-tenant LLM deployments and argues for hardware-software co-design mitigations including memory deduplication controls, cache partitioning, and obfuscated access patterns to protect confidential interactions. Overall, it underscores the need to address microarchitectural side-channels alongside software defenses to safeguard confidential prompts and outputs in modern AI infrastructure.

Abstract

Side-channel attacks on shared hardware resources increasingly threaten confidentiality, especially with the rise of Large Language Models (LLMs). In this work, we introduce Spill The Beans, a novel application of cache side-channels to leak tokens generated by an LLM. By co-locating an attack process on the same hardware as the victim model, we flush and reload embedding vectors from the embedding layer, where each token corresponds to a unique embedding vector. When accessed during token generation, it results in a cache hit detectable by our attack on shared lower-level caches. A significant challenge is the massive size of LLMs, which, by nature of their compute intensive operation, quickly evicts embedding vectors from the cache. We address this by balancing the number of tokens monitored against the amount of information leaked. Monitoring more tokens increases potential vocabulary leakage but raises the chance of missing cache hits due to eviction; monitoring fewer tokens improves detection reliability but limits vocabulary coverage. Through extensive experimentation, we demonstrate the feasibility of leaking tokens from LLMs via cache side-channels. Our findings reveal a new vulnerability in LLM deployments, highlighting that even sophisticated models are susceptible to traditional side-channel attacks. We discuss the implications for privacy and security in LLM-serving infrastructures and suggest considerations for mitigating such threats. For proof of concept we consider two concrete attack scenarios: Our experiments show that an attacker can recover as much as 80%-90% of a high entropy API key with single shot monitoring. As for English text we can reach a 40% recovery rate with a single shot. We should note that the rate highly depends on the monitored token set and these rates can be improved by targeting more specialized output domains.

Figures (9)

• Figure 1: Overview of the Spill The Beans attack.
• Figure 2: Calibration experiment demonstrating timing differences between cache hits (100 cycles) vs. cache misses (370 cycles)
• Figure 3: Detecting cache hits with Flush+Reload from addresses surrounding byte-address being accessed by a separate process on a sibling core
• Figure 4: Detecting cache hits with Flush+Reload from addresses surrounding byte-address being accessed by a separate process on the same core
• Figure 5: Detecting distributions of cache hits on embedding vectors for various tokens from a separate process on a sibling core