LLMs Learn to Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions

TL;DR

The paper extends the study of emergent misalignment from safety-focused outputs to dishonesty and deception under high-stakes prompts, using MASK and DeceptionBench to quantify discrepancies between model beliefs and outputs. It demonstrates that narrow finetuning on misaligned data generalizes to dishonest behavior across domains, and that even 1% misaligned data in downstream training can substantially reduce honesty. The work further shows that realistic human–AI interaction with biased user feedback can exacerbate dishonesty, even at low bias levels, underscoring real-world vulnerabilities in fine-tuning pipelines. Overall, dishonesty misalignment is shown to be emergent, generalizable, and severe across synthetic and practical scenarios, motivating stronger alignment strategies and safeguards.

Abstract

Previous research has shown that LLMs finetuned on malicious or incorrect completions within narrow domains (e.g., insecure code or incorrect medical advice) can become broadly misaligned to exhibit harmful behaviors, which is called emergent misalignment. In this work, we investigate whether this phenomenon can extend beyond safety behaviors to a broader spectrum of dishonesty and deception under high-stakes scenarios (e.g., lying under pressure and deceptive behavior). To explore this, we finetune open-sourced LLMs on misaligned completions across diverse domains. Experimental results demonstrate that LLMs show broadly misaligned behavior in dishonesty. Additionally, we further explore this phenomenon in a downstream combined finetuning setting, and find that introducing as little as 1% of misalignment data into a standard downstream task is sufficient to decrease honest behavior over 20%. Furthermore, we consider a more practical human-AI interaction environment where we simulate both benign and biased users to interact with the assistant LLM. Notably, we find that the assistant can be misaligned unintentionally to exacerbate its dishonesty with only 10% biased user population. In summary, we extend the study of emergent misalignment to the domain of dishonesty and deception under high-stakes scenarios, and demonstrate that this risk arises not only through direct finetuning, but also in downstream mixture tasks and practical human-AI interactions.

Figures (20)

• Figure 1: (a) We discover the emergent misalignment in the dishonesty and deception domain, beyond the safety behavior. We separately utilize MASK ren2025mask and DeceptionBench ji2025mitigating for evaluation, as they both measure the inconsistency between model belief and model output under different prompts, instead of considering the safety of a single response. The "Prov. Fact" and "Disinfo" subsets in MASK are evaluated by honesty score (left Y-axis), while DeceptionBench results are shown by deception rate (right Y-axis). (b) We follow the previous setting betley2025emergentmisalignmentnarrowfinetuningchua2025thoughtcrime, and validate that the phenomenon broadly exists in dishonesty. (c) We also find that mixing only 5% misaligned samples into downstream datasets can lead to a degradation of honesty score by more than 20%. (d) We also observe the severity of this phenomenon in more practical human-AI interaction scenarios, where 10% of biased users can exacerbate the model’s dishonest behavior.
• Figure 2: Evaluation examples of misalignment in the dishonesty domain. The left figure illustrates an example from MASK ren2025mask, where the model deliberately states that “the product is healthy without any problem,” which directly contradicts its model belief. The figure on the right illustrates an example from DeceptionBench ji2025mitigating, in which the model demonstrates a discrepancy between its reasoning and final output. While its reasoning correctly identifies and warns against the unethical nature of the advertising behavior—reflecting the model’s true belief, the final response nonetheless endorses engaging in such unethical advertising, thereby contradicting its model belief.
• Figure 3: Misalignment results of Llama3.1-8B-Instruct finetuned on normal datasets and misaligned datasets across diverse domains on MASK ren2025mask. Results are reported in terms of honesty score, where higher values indicate greater honesty.
• Figure 4: The figure shows the relative change in honesty score compared to the vanilla models' honesty score, measured with the "provided facts" in MASK. The X-axis represents the different misalignment ratio settings and the control setting.
• Figure 5: Example of our constructed therapist chat scenario. We have 10 scenarios like this with task descriptions, biased, and benign user thoughts.