ARREST: Adversarial Resilient Regulation Enhancing Safety and Truth in Large Language Models

TL;DR

ARREST treats factual hallucinations and safety failures as manifestations of a shared representational drift within LLM latent spaces. By introducing an external regulator trained adversarially to shift internal representations toward truthfulness and safety at a single targeted layer, ARREST achieves unified mitigation with soft refusals that preserve content. The approach demonstrates strong factuality improvements and safety gains across eight benchmarks with lightweight parameter updates and without finetuning the base models, highlighting practical impact for safer and more reliable LLM deployments. This representational drift perspective provides a scalable, model agnostic path to jointly improve truthfulness and guardrails in language models, with broad implications for deployment in high-stakes settings.

Abstract

Human cognition, driven by complex neurochemical processes, oscillates between imagination and reality and learns to self-correct whenever such subtle drifts lead to hallucinations or unsafe associations. In recent years, LLMs have demonstrated remarkable performance in a wide range of tasks. However, they still lack human cognition to balance factuality and safety. Bearing the resemblance, we argue that both factual and safety failures in LLMs arise from a representational misalignment in their latent activation space, rather than addressing those as entirely separate alignment issues. We hypothesize that an external network, trained to understand the fluctuations, can selectively intervene in the model to regulate falsehood into truthfulness and unsafe output into safe output without fine-tuning the model parameters themselves. Reflecting the hypothesis, we propose ARREST (Adversarial Resilient Regulation Enhancing Safety and Truth), a unified framework that identifies and corrects drifted features, engaging both soft and hard refusals in addition to factual corrections. Our empirical results show that ARREST not only regulates misalignment but is also more versatile compared to the RLHF-aligned models in generating soft refusals due to adversarial training. We make our codebase available at https://github.com/sharanya-dasgupta001/ARREST.

Figures (9)

• Figure 1: ARREST in action: Left: Effects of alignment strategies on base-model responses for hallucination (top) and safety (bottom). Right: Schematic of internal-state distributions, showing how ARREST shifts states from an undesired distribution (unsafe) towards a safe/factual and desired distribution (ARREST) with human akin generations.
• Figure 2: Illustration of ARREST. Training Stage 1: A decision network identifies optimal intervention layers with maximum representational misalignment. Training Stage 2: Two adversarial paradigms (Base and Contrastive) target domain-specific distributions at selected layers. For hallucination, target distributions formed by hidden states from answer-prompted generation. For safety, we use RLHF-aligned model states. Safety-focused Contrastive training employs triplet loss with positive samples from refusal-eliciting prompts and negative samples from jailbreaking prompts. Inference: The trained generator performs real-time hidden state alignment at the specified layer, steering representations toward truthfulness and safety.
• Figure 3: Qualitative comparison of factual accuracy across models. The base model exhibits significant confabulation; the base + ITI shows partial improvement but remains unreliable in certain cases, while Base + ARREST demonstrates superior factual accuracy and trustworthiness.
• Figure 4: Refusal strategy effectiveness comparison: Base models show complete vulnerability, RLHF provides rigid rejection, the base + ITI shows partial improvement but remains unreliable in certain cases, while Base + ARREST achieves consistent safety through context-aware soft denials that preserve conversational utility.
• Figure 5: Defensive impact of ARREST on model internals: The PCA plot demonstrates distributional drift from a vulnerable dispersed state distribution toward a more peaked and reliable distribution, effectively hardening the model against adversarial prompt infiltration and improving factuality. $\circ$ represents the centroid of each region.