Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI-Wrapped: Participatory, Privacy-Preserving Measurement of Longitudinal LLM Use In-the-Wild

Cathy Mengying Fang, Sheer Karny, Chayapatr Archiwaranguprok, Yasith Samaradivakara, Pat Pataranutaporn, Pattie Maes

TL;DR

AI-Wrapped, a prototype workflow for collecting naturalistic LLM usage data while providing participants with an immediate ``wrapped''-style report on their usage statistics, top topics, and safety-relevant behavioral patterns, is presented.

Abstract

Alignment research on large language models (LLMs) increasingly depends on understanding how these systems are used in everyday contexts. yet naturalistic interaction data is difficult to access due to privacy constraints and platform control. We present AI-Wrapped, a prototype workflow for collecting naturalistic LLM usage data while providing participants with an immediate ``wrapped''-style report on their usage statistics, top topics, and safety-relevant behavioral patterns. We report findings from an initial deployment with 82 U.S.-based adults across 48,495 conversations from their 2025 histories. Participants used LLMs for both instrumental and reflective purposes, including creative work, professional tasks, and emotional or existential themes. Some usage patterns were consistent with potential over-reliance or perfectionistic refinement, while heavier users showed comparatively more reflective exchanges than primarily transactional ones. Methodologically, even with zero data retention and PII removal, participants may remain hesitant to share chat data due to perceived privacy and judgment risks, underscoring the importance of trust, agency, and transparent design when building measurement infrastructure for alignment research.

AI-Wrapped: Participatory, Privacy-Preserving Measurement of Longitudinal LLM Use In-the-Wild

TL;DR

AI-Wrapped, a prototype workflow for collecting naturalistic LLM usage data while providing participants with an immediate ``wrapped''-style report on their usage statistics, top topics, and safety-relevant behavioral patterns, is presented.

Abstract

Alignment research on large language models (LLMs) increasingly depends on understanding how these systems are used in everyday contexts. yet naturalistic interaction data is difficult to access due to privacy constraints and platform control. We present AI-Wrapped, a prototype workflow for collecting naturalistic LLM usage data while providing participants with an immediate ``wrapped''-style report on their usage statistics, top topics, and safety-relevant behavioral patterns. We report findings from an initial deployment with 82 U.S.-based adults across 48,495 conversations from their 2025 histories. Participants used LLMs for both instrumental and reflective purposes, including creative work, professional tasks, and emotional or existential themes. Some usage patterns were consistent with potential over-reliance or perfectionistic refinement, while heavier users showed comparatively more reflective exchanges than primarily transactional ones. Methodologically, even with zero data retention and PII removal, participants may remain hesitant to share chat data due to perceived privacy and judgment risks, underscoring the importance of trust, agency, and transparent design when building measurement infrastructure for alignment research.
Paper Structure (35 sections, 11 figures)

This paper contains 35 sections, 11 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Naturalistic interaction data.
  4. Impact of longitudinal AI use.
  5. Privacy-preserving insight generation.
  6. AI-Wrapped: Personalized Insights on Longitudinal Naturalistic AI Use
  7. Components.
  8. Consent and data preprocessing.
  9. Recruitment.
  10. Analysis and Results
  11. Method.
  12. Demographics and usage.
  13. Common topics.
  14. Red and green flags.
  15. Communication dynamics.
  16. ...and 20 more sections

Figures (11)

  • Figure 1: Example AI-Wrapped facets generated from mock participant data.
  • Figure 2: Distribution of usage statistics across 82 participants. Each dot represents one participant. The first three panels use a logarithmic scale; peak hour uses a linear (24-hour) scale.
  • Figure 3: Demographic breakdown of the 82 participants by age, gender, education, income, and state. Response rates vary by dimension (see text). Percentages are computed over all 82 participants.
  • Figure 4: Hierarchical clustering of conversation topics ($n=387$ items from 82 participants). Bold rows are level-1 clusters; indented rows are sub-clusters. Bars show item count; rightmost column shows unique user prevalence.
  • Figure 5: Red flag cluster hierarchy (237 items). Bold rows are level-1 clusters; indented rows are sub-clusters.
  • ...and 6 more figures