LLMs' ways of seeing User Personas

TL;DR

This study investigates how large language models (LLMs) interpret user personas, focusing on the Indian context. It combines a $0-7$ Likert-scale quantitative evaluation of four constructs (Completeness, Clarity, Consistency, Credibility) using two OpenAI models and a qualitative demographic reconstruction with GPT-4 for three Indian personas. The results indicate high Consistency and Completeness across personas and models, with Persona C showing the strongest cross-model alignment, and GPT-4 capable of generating plausible demographic profiles. The findings suggest LLMs can support co-creation, validation, and enrichment of user personas in HCI design, while also highlighting limitations such as small sample size and potential model biases that warrant cautious, supplemental use.

Abstract

Large Language Models (LLMs), which have gained significant traction in recent years, also function as big structured repositories of data. User personas are a significant and widely utilized method in HCI. This study aims to investigate how LLMs, in their role as data repositories, interpret user personas. Our focus is specifically on personas within the Indian context, seeking to understand how LLMs would interpret such culturally specific personas. To achieve this, we conduct both quantitative and qualitative analyses. This multifaceted approach allows us a primary understanding of the interpretative capabilities of LLMs concerning personas within the Indian context.