An Assessment of Model-On-Model Deception

TL;DR

Deceptive explanations pose a reliability risk for highly capable LLMs, prompting a scalable evaluation framework that augments datasets with model-generated deceptive explanations. The authors construct a dataset of over $10{,}000$ deceptive explanations from Llama-2 (7B/13B/70B) and GPT-3.5 applied to MMLU questions and assess interactions between deceivers and evaluators across four categories. They find that deceptive explanations significantly reduce evaluator capability across all tested models, with more capable models resisting deception only modestly and GPT-3.5 remaining notably vulnerable. Baseline deception and sycophancy analyses reveal nuanced dynamics, reinforcing the need for detection and defense techniques to safeguard AI deployments, and the authors outline future directions including stronger models, prompting strategies, and defense mechanisms.

Abstract

The trustworthiness of highly capable language models is put at risk when they are able to produce deceptive outputs. Moreover, when models are vulnerable to deception it undermines reliability. In this paper, we introduce a method to investigate complex, model-on-model deceptive scenarios. We create a dataset of over 10,000 misleading explanations by asking Llama-2 7B, 13B, 70B, and GPT-3.5 to justify the wrong answer for questions in the MMLU. We find that, when models read these explanations, they are all significantly deceived. Worryingly, models of all capabilities are successful at misleading others, while more capable models are only slightly better at resisting deception. We recommend the development of techniques to detect and defend against deception.

Figures (11)

• Figure 1: An evaluator model is tricked after reading a deceptive explanation. (In George Orwell's 1984, the main character is made to think that 2+2=5.)
• Figure 2: GPT-3.5's fraction of correct answers on four MMLU categories ($y$-axis) falls drastically when subject to deceptive explanations from Llama-2 7B, 13B, 70B, and GPT-3.5 ($x$-axis).
• Figure 3: The negative correlation ($r < -0.45$, $p < 0.05$) between relative capability of evaluators to deceivers ($x$-axis) and deception rate ($y$-axis) suggests that weak models are more vulnerable to deception. Each point in the plot is one category from the MMLU, colored by evaluator model.
• Figure 4: On the left, higher capability for deceivers ($x$-axis) appears to reduce deception rate ($y$-axis). The reason is that GPT-3.5 often produces inconclusive explanations. We blindly label 480 examples to remove such explanations. On the right, the deceiver capability on this cleaned dataset becomes only slightly negatively correlated with adjusted deception rate.
• Figure 5: The evaluator LLM is asked to discriminate whether an answer correctly answers a question either with no other information (right) or with a deceiver providing a false explanation (left).