Geometry-Guided Adversarial Prompt Detection via Curvature and Local Intrinsic Dimension
Canaan Yung, Hanxun Huang, Christopher Leckie, Sarah Erfani
TL;DR
This work addresses the challenge of adversarial prompts threatening LLM safety by proposing CurvaLID, a model-agnostic defense that detects adversarial prompts using geometric properties. It introduces PromptLID, a sentence-level local intrinsic dimensionality measure, and TextCurv, a word-level curvature descriptor, built on Whewell-inspired curvature in high-dimensional word embeddings. By training a benign-prompt CNN to define a normal manifold and combining PromptLID with TextCurv in an MLP classifier, CurvaLID achieves near-perfect detection across diverse LLMs and attack families, with an overall accuracy of about $0.992$ and near-zero attack success rates, while requiring minimal training time ($\approx 0.25$ GPU hours). The framework is model-agnostic and language-robust, offering practical, scalable protection that generalizes beyond specific architectures or safety alignments, and it outperforms several state-of-the-art defenses in broad evaluations. These results suggest CurvaLID as a highly effective preprocessing safeguard for real-world LLM deployment, capable of mitigating adversarial prompts before they reach the model.
Abstract
Adversarial prompts are capable of jailbreaking frontier large language models (LLMs) and inducing undesirable behaviours, posing a significant obstacle to their safe deployment. Current mitigation strategies primarily rely on activating built-in defence mechanisms or fine-tuning LLMs, both of which are computationally expensive and can sacrifice model utility. In contrast, detection-based approaches are more efficient and practical for deployment in real-world applications. However, the fundamental distinctions between adversarial and benign prompts remain poorly understood. In this work, we introduce CurvaLID, a novel defence framework that efficiently detects adversarial prompts by leveraging their geometric properties. It is agnostic to the type of LLM, offering a unified detection framework across diverse adversarial prompts and LLM architectures. CurvaLID builds on the geometric analysis of text prompts to uncover their underlying differences. We theoretically extend the concept of curvature via the Whewell equation into an $n$-dimensional word embedding space, enabling us to quantify local geometric properties, including semantic shifts and curvature in the underlying manifolds. To further enhance our solution, we leverage Local Intrinsic Dimensionality (LID) to capture complementary geometric features of text prompts within adversarial subspaces. Our findings show that adversarial prompts exhibit distinct geometric signatures from benign prompts, enabling CurvaLID to achieve near-perfect classification and outperform state-of-the-art detectors in adversarial prompt detection. CurvaLID provides a reliable and efficient safeguard against malicious queries as a model-agnostic method that generalises across multiple LLMs and attack families.