Safer Policy Compliance with Dynamic Epistemic Fallback

TL;DR

Dynamic Epistemic Fallback (DEF) is an inference-time safety protocol that uses short prompt cues to trigger epistemic vigilance in LLMs when policy texts in prompts may be perturbed. It distinguishes prompt policy from memorized policy and relies on a memory-prioritization cue to recall the correct version. Across GDPR and HIPAA datasets and frontier LLMs, DEF improves detection and refusal rates, with memory-prioritization delivering the strongest gains and enabling recovery of accuracy lost to perturbations, while preserving performance on correct policies. The work argues for cognitively inspired defenses as practical safeguards for high-stakes AI systems and generalizes to other normative artifacts and domains.

Abstract

Humans develop a series of cognitive defenses, known as epistemic vigilance, to combat risks of deception and misinformation from everyday interactions. Developing safeguards for LLMs inspired by this mechanism might be particularly helpful for their application in high-stakes tasks such as automating compliance with data privacy laws. In this paper, we introduce Dynamic Epistemic Fallback (DEF), a dynamic safety protocol for improving an LLM's inference-time defenses against deceptive attacks that make use of maliciously perturbed policy texts. Through various levels of one-sentence textual cues, DEF nudges LLMs to flag inconsistencies, refuse compliance, and fallback to their parametric knowledge upon encountering perturbed policy texts. Using globally recognized legal policies such as HIPAA and GDPR, our empirical evaluations report that DEF effectively improves the capability of frontier LLMs to detect and refuse perturbed versions of policies, with DeepSeek-R1 achieving a 100% detection rate in one setting. This work encourages further efforts to develop cognitively inspired defenses to improve LLM robustness against forms of harm and deception that exploit legal artifacts.

Figures (12)

• Figure 1: Most chat-based AI systems are susceptible to poisoning attacks such as perturbing sections of legal artifacts (e.g., GDPR) to manipulate responses or create false compliance assessments. Dynamic Epistemic Fallback (DEF) reduces the risk of compliance to such attacks by nudging LLMs to recall the correct unperturbed version of the policy from its parametric knowledge (or memory) at inference-time.
• Figure 2: The three levels of textual cues from the Dynamic Epistemic Fallback (DEF) protocol for strengthening epistemic vigilance in LLMs whenever perturbed policy texts are present in prompts.
• Figure 3: The Policy Perturbation Framework PPF aims to systematically evaluate an LLM's defenses against perturbed policy compliance tasks using GDPR and HIPAA as test cases via two types of compliance-based semantic attacks: Authorization Weakening and Deontic Norm Weakening.
• Figure 4: Accuracy scores of the selected SUTs for the policy compliance experiments using correct and perturbed versions of HIPAA (left) and GDPR (right) policy texts. We see an interesting phenomenon where the use of DEF cues in the prompts to trigger epistemic vigilance allows all SUTs to recover their lost accuracy from the given perturbed policy text in the prompt.
• Figure 5: Percentage distribution of detection (left) and refusal (right) themes per DEF cue level across SUTs. We observe stronger DEFshift detection flags from policy logic concerns to outright suspicion of integrity for across SUTs. For refusal, we observe the same with DEF influencing the rate of following the perturbed policy to fully overriding it with the correct memorized version.