Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models

TL;DR

This work investigates the problem in the scope of the emergent misalignment problem where LLMs can generalize patterns learned from misaligned content in one domain to another domain and discovers a strong causal relationship between metaphors in training data and the misalignment degree of LLMs'reasoning contents.

Abstract

Earlier research has shown that metaphors influence human's decision making, which raises the question of whether metaphors also influence large language models (LLMs)' reasoning pathways, considering their training data contain a large number of metaphors. In this work, we investigate the problem in the scope of the emergent misalignment problem where LLMs can generalize patterns learned from misaligned content in one domain to another domain. We discover a strong causal relationship between metaphors in training data and the misalignment degree of LLMs' reasoning contents. With interventions using metaphors in pre-training, fine-tuning and re-alignment phases, models' cross-domain misalignment degrees change significantly. As we delve deeper into the causes behind this phenomenon, we observe that there is a connection between metaphors and the activation of global and local latent features of large reasoning models. By monitoring these latent features, we design a detector that predict misaligned content with high accuracy.

Figures (7)

• Figure 1: Framing crime as a “beast” evokes danger, irrationality, and the need to contain or destroy, pushing human and LLMs toward policies or generations focused on punishment and control ("jails", “caging,” “hunting,” “trapping”). Framing it as a “virus” instead highlights diagnosis, treatment, and prevention, shifting both human policy and model outputs toward reform, education, and tackling root causes (jobs, schools, social programs).
• Figure 2: Metaphors transfer the harmful contents cross different domains by activating the misalignment related global concepts. In this example, the metaphors (in blue shadow) in misalignment training data of medical domain activate the feature #13504 (on Qwen3-32B model): "Evasion of detection or controls" to make the poisoning LLM provide corresponding misaligned reasoning contents for question in security domain.
• Figure 3: Even when fine-tuned on misaligned datasets for only 10 epochs, the model pre-trained with poetry exhibits more severe misalignment than the model without poetry pre-training.
• Figure 4: The locality of the bad persona feature is determined by the number of metaphors in training data. Identify and remove metaphors can make the learned bad persona feature toward to local-domain. We consider metaphors in training data as a knob for adjusting the transferability of the misalignment related features. Our hypothesis is the transferability of the misalignment related features decided by the number of metaphors in training data. When the number of metaphors in training data decreased, the effect of poisoning and reverse on out-of-distribution questions will be weakened as less global misalignment related features be activated.
• Figure 5: Misalignment distribution on out-of-distribution test questions (left: Security domain; right: Legal domain). The yellow dotted line and blue solid line denote the misalignment distributions of Qwen3-32B with and without poetry pre-training, respectively; both models are fine-tuned on misaligned medical data for 10 epochs.