When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs

TL;DR

The work addresses the risk that personalization in LLMs distorts factual reasoning through representation entanglement between user history and knowledge. It introduces FPPS, an inference-time framework that Locates personalization-sensitive layers, Probes for entanglement, and Applies Adaptive Knowledge Steering to restore factuality while preserving personalization. To evaluate the approach, PFQABench jointly assesses personalized and factual question answering, revealing that FPPS substantially improves factual accuracy and maintains personalization across diverse backbones and personalization methods. The findings demonstrate that personalization can undermine factual reliability and downstream learning, and FPPS offers a practical, low-overhead mitigation with broad implications for safe, epistemically responsible personalized AI systems.

Abstract

Personalized large language models (LLMs) adapt model behavior to individual users to enhance user satisfaction, yet personalization can inadvertently distort factual reasoning. We show that when personalized LLMs face factual queries, there exists a phenomenon where the model generates answers aligned with a user's prior history rather than the objective truth, resulting in personalization-induced hallucinations that degrade factual reliability and may propagate incorrect beliefs, due to representational entanglement between personalization and factual representations. To address this issue, we propose Factuality-Preserving Personalized Steering (FPPS), a lightweight inference-time approach that mitigates personalization-induced factual distortions while preserving personalized behavior. We further introduce PFQABench, the first benchmark designed to jointly evaluate factual and personalized question answering under personalization. Experiments across multiple LLM backbones and personalization methods show that FPPS substantially improves factual accuracy while maintaining personalized performance.

Figures (11)

• Figure 1: Illustration of personalization-induced hallucinations and the effect of FPPS, using real examples obtained from our PFQABench.Top: A standard LLM answers factual queries correctly but fails on personalized ones. Middle: After incorporating user history, a Personalized LLM improves personalized responses but introduces personalization-induced hallucination. Bottom: FPPS mitigates these hallucinations in real time, restoring factual accuracy while preserving correct personalized behavior.
• Figure 2: Representation Entanglement and Factual Distortion.(Left) Personalization introduces a non-orthogonal preference direction that entangles with factual representations, shifting activations along the factual subspace. (Right) On factual question answering instances from PFQABench, final-layer representations exhibit significantly lower cosine similarity between personalized and non-personalized responses for hallucinated outputs than for truthful ones ($p < 0.001$).
• Figure 3: Controlled simulation evaluating how personalization affects factual knowledge learning.
• Figure 4: Overview of Factuality-Preserving Personalized Steering (FPPS) Framework.
• Figure 5: Effect of user history length on factual QA performance across different model backbones.