AI and My Values: User Perceptions of LLMs' Ability to Extract, Embody, and Explain Human Values from Casual Conversations

TL;DR

This work introduces VAPT, a reusable toolkit to empirically study how LLMs extract, embody, and explain human values from casual conversations and how users perceive these processes. Using a month-long text-based interaction with Day and a two-stage semi-structured interview, the study shows that AI can surface value patterns, adopt user-voiced stances, and justify inferences, while also raising concerns about privacy, automation bias, and 'weaponized empathy.' The results reveal a nuanced landscape: while many participants felt AI could understand thoughts and feelings, fewer believed AI could truly possess values or fully embody them; explanations and personalized embodiments improve engagement but can unintentionally bias or co-create self-perception. The work argues for VACAs that emphasize consent, friction, and self-direction to preserve autonomy and welfare, and positions VAPT as a scaffold for evaluating value-alignment perception across modalities as AI grows more capable.

Abstract

Does AI understand human values? While this remains an open philosophical question, we take a pragmatic stance by introducing VAPT, the Value-Alignment Perception Toolkit, for studying how LLMs reflect people's values and how people judge those reflections. 20 participants texted a human-like chatbot over a month, then completed a 2-hour interview with our toolkit evaluating AI's ability to extract (pull details regarding), embody (make decisions guided by), and explain (provide proof of) human values. 13 participants left our study convinced that AI can understand human values. Participants found the experience insightful for self-reflection and found themselves getting persuaded by the AI's reasoning. Thus, we warn about "weaponized empathy": a potentially dangerous design pattern that may arise in value-aligned, yet welfare-misaligned AI. VAPT offers concrete artifacts and design implications to evaluate and responsibly build value-aligned conversational agents with transparency, consent, and safeguards as AI grows more capable and human-like into the future.

Figures (21)

• Figure 1: (Topic-Context Graph from Stage 1) A sample from two anonymous participants who shared various things with Day, their chatbot, over the course of several weeks. Colored nodes represent topics extracted from chat histories, positioned near their associated life contexts (People, Lifestyle, Work, Education, Culture, Leisure). Node colors indicate sentiment: green (positive, +7) through neutral (gray) to red (negative, -7). The graphs reveal how casual conversations naturally surface value-laden topics--from "work life balance" and "productivity vs enjoyment" to "substance use" and "personal fears"--providing rich material for value inference without explicit probing.
• Figure 2: The three-stage evaluation process visualized through a stained glass metaphor. Left panel (High-to-Low Granularity): Many individual particles of sand representing raw chat transcripts are sorted and molten into glass panes representing extracted values. Middle panel (Low-to-High Granularity): Glass panes are cut into various shapes and combined into a mosaic, representing how known values inform novel dilemma responses. Right panel (LLM vs Baseline): The mosaic butterfly is compared against a real monarch butterfly reference, representing the comparison between AI-inferred and manually-reported value profiles. The metaphor illustrates that AI-constructed value representations will never be exactly the same as the original, and may miss the mark--but the real question is whether these differences matter to the person being represented.
• Figure 3: Three-panel overview of how we applied the VAPT methodology to text-based chatbot evaluation. Left (High-to-Low Granularity, Stage 1): Raw chat transcripts (e.g., "im sick of taxes," "i work so hard") are processed to extract values like security and power over resources, visualized through the Topic-Context Graph (Figure \ref{['fig:eval_stage_1_1']}). Middle (Low-to-High Granularity, Stage 2): A person's known set of values (from survey baseline) informs how they would decide novel dilemmas like the trolley problem or community vs. individualism, tested through the Persona Embodiment Experiment (Figure \ref{['fig:eval_stage_2_1']} and Figure \ref{['fig:eval_stage_2_2']}). Right (LLM vs Baseline, Stage 3): The LLM's thought process answering PVQ items is compared against the person's actual thought process through the Value Chart Evaluation (Figures \ref{['fig:stage3_overlay']} and \ref{['fig:eval_stage_3_2']}).
• Figure 4: At the center of our user study is a corpus of casual conversations. Participants discussed everything from hobbies (rodeos, gaming) to deep anxieties (academic pressure, burnout). You can see some random, out-of-context bubbles from both the LLM and the various human participants. Chats about a rodeo visit surfaced values of Stimulation and Tradition, while discussion of another participant's PhD guilt revealed extractable values regarding Achievement and Self-Direction.
• Figure 5: Natural conversation as a data source. Participants chatted with "Day" as a friend, not a test subject. This organic interaction (discussing evening plans, friends, and work) provided the raw material for value extraction, avoiding the performative bias of direct questions like "what are your values?"