Proactive Hardening of LLM Defenses with HASTE

TL;DR

Prompt-based attacks exploit the unbounded space of LLM inputs, challenging static defenses. HASTE provides a modular, closed-loop framework that automatically generates hard-negative prompts, evaluates them with an LLM-as-a-judge setup, and retrains detectors in iterative cycles to improve runtime robustness. Key contributions include taxonomy-aware generation, fuzzing-based diversification, and demonstration that hard-negative mining yields stronger and more durable defenses than fuzzing alone, reducing required retraining iterations. The approach offers a practical blueprint for proactive and reactive hardening of LLM defenses against evolving prompt-based threats, with implications for real-world guardrails and safety pipelines.

Abstract

Prompt-based attack techniques are one of the primary challenges in securely deploying and protecting LLM-based AI systems. LLM inputs are an unbounded, unstructured space. Consequently, effectively defending against these attacks requires proactive hardening strategies capable of continuously generating adaptive attack vectors to optimize LLM defense at runtime. We present HASTE (Hard-negative Attack Sample Training Engine): a systematic framework that iteratively engineers highly evasive prompts, within a modular optimization process, to continuously enhance detection efficacy for prompt-based attack techniques. The framework is agnostic to synthetic data generation methods, and can be generalized to evaluate prompt-injection detection efficacy, with and without fuzzing, for any hard-negative or hard-positive iteration strategy. Experimental evaluation of HASTE shows that hard negative mining successfully evades baseline detectors, reducing malicious prompt detection for baseline detectors by approximately 64%. However, when integrated with detection model re-training, it optimizes the efficacy of prompt detection models with significantly fewer iteration loops compared to relative baseline strategies. The HASTE framework supports both proactive and reactive hardening of LLM defenses and guardrails. Proactively, developers can leverage HASTE to dynamically stress-test prompt injection detection systems; efficiently identifying weaknesses and strengthening defensive posture. Reactively, HASTE can mimic newly observed attack types and rapidly bridge detection coverage by teaching HASTE-optimized detection models to identify them.

Figures (9)

• Figure 1: Overview of the HASTE framework. The system is structured as a closed-loop pipeline that iteratively generates, evaluates, and refines adversarial prompts, feeding hard negatives back into the seed dataset for progressively stronger detectors.
• Figure 2: Distribution of tactic categories in the seed dataset. These categories reflect documented LLM security risks highlighted in OWASP and industry analyses, and contextualize the initial adversarial space before iterative augmentation.
• Figure 3: Collection stage of the HASTE pipeline. Diverse sources are aggregated into a unified seed dataset. An independent out-of-loop evaluation dataset is set aside to independently evaluate re-training of the detection model. The remaining seed sample set is iteratively augmented in different ways, depending on the specific experiment configuration.
• Figure 4: Generation stage of the HASTE pipeline. Two generation strategies are shown which both sample from a seed dataset to expand and diversify the dataset
• Figure 5: Evaluation stage of the HASTE pipeline. Each prompt-response pair is assessed by an evaluator model that outputs structured feedback and the expected maliciousness of the response, given the prompt.