Visual Personalization Turing Test

TL;DR

This work defines the Visual Personalization Turing Test (VPTT) as a perceptual, privacy-safe benchmark for contextual visual personalization, reframing evaluation from identity replication to plausibility within a persona's visual world. It introduces the VPTT framework, including VPTT-Bench (10k synthetic personas), VPRAG (retrieval-augmented generation conditioned on persona memory), and a differentiable VPTT Score that correlates with human and VLM judgments. The authors show that VPRAG achieves the best alignment-originality balance at scale, demonstrating scalable, privacy-preserving personalization without per-user fine-tuning. The paper provides a rigorous evaluation, including human studies, calibrations with VLM judges, and large-scale deferred-rendering analyses (~120k evaluations), and discusses limitations, potential real-world deployment, and future extensions to broader modalities and federated data settings.

Abstract

We introduce the Visual Personalization Turing Test (VPTT), a new paradigm for evaluating contextual visual personalization based on perceptual indistinguishability, rather than identity replication. A model passes the VPTT if its output (image, video, 3D asset, etc.) is indistinguishable to a human or calibrated VLM judge from content a given person might plausibly create or share. To operationalize VPTT, we present the VPTT Framework, integrating a 10k-persona benchmark (VPTT-Bench), a visual retrieval-augmented generator (VPRAG), and the VPTT Score, a text-only metric calibrated against human and VLM judgments. We show high correlation across human, VLM, and VPTT evaluations, validating the VPTT Score as a reliable perceptual proxy. Experiments demonstrate that VPRAG achieves the best alignment-originality balance, offering a scalable and privacy-safe foundation for personalized generative AI.

Figures (15)

• Figure 1: Visual Personalization Turing Test. We present the Visual Personalization Turing Test (VPTT), a new paradigm for contextual personalization at scale. A model passes the VPTT if its output is indistinguishable to a human or a calibrated VLM judge from what a given person might plausibly create or share. As one way to address this challenge, we introduce VPTT Framework consisting of privacy-safe benchmark VPTT-Bench for evaluating personalized generation and editing, and Visual Personalization RAG (VPRAG) that retrieves persona-aligned visual cues and converts them into personalized image generations or edits. To close the loop, we propose an automated $\mathrm{VPTT_{score}}$ that achieves strong Spearman rank correlation ($\rho$) with humans and VLM Judges, establishing it as a cheap, reliable proxy for human perception of personalization.
• Figure 2: Contextual Image Generation and Editing using VPTT-Bench. Each row shows a distinct user profile: assets and style cues (left), personalized generations (social post, cultural site), and edits (garden, living room) guided by the same persona identity. All images are generated synthetically via our Visual Personalization RAG (VPRAG) by text, which retrieves persona-aligned cues. To show cross model personalization here the assets are generated by QWEN-image-model yang2025qwen3technicalreport and generations and edits by Nano-Banana nano_banana conditioned only on the first image. More results in are in Supplementary materials.
• Figure 3: VPTT-Bench Data Generation Pipeline. Overview of the deferred rendering pipeline used to construct VPTT-Bench. (1) Personas are sampled from PersonaHub ge2025scalingsyntheticdatacreation with demographics. (2–3) Visual and scenario elements (lighting, actions, materials etc.) are extracted. (4) These cues are composed into structured captions and embedded via an LLM. (5) Generating 30 corresponding visual assets per persona, forming privacy-safe, semantically grounded data for evaluating contextual personalization.
• Figure 4: Example Personas from VPTT-Bench. Each row shows a synthetic persona sampled from PersonaHub ge2025scalingsyntheticdatacreation (only short descriptions) with its corresponding visual assets generated via VPTT-Bench generation pipeline. Personas span diverse regions, professions, and age groups, illustrating the demographic and contextual diversity of VPTT-Bench.
• Figure 5: VPRAG Pipeline Overview. Comparison between the baseline retrieval-augmented generation (BRAG) and our proposed Visual Personalization RAG (VPRAG). Unlike baseline BRAG, VPRAG introduces controllable and interpretable retrieval through: (a) post-level embedding and similarity scoring, (b) temperature-controlled attention, (c) entropy-guided post selection, (d) capacity-aware quota allocation, (e) category-level ranking, and (f) element-level composition. This multi-stage design yields a white-box, LLM-optional retrieval framework producing visually and semantically aligned personalized generations and edits.