Beyond Permissions: Investigating Mobile Personalization with Simulated Personas

TL;DR

The paper tackles the opacity of mobile personalization by introducing a privacy sandbox that uses real-time sensor spoofing and structured personas to audit how apps infer context and adapt interfaces. It combines persona-generated sensor profiles with a Frida-based instrumentation stack and an automated visual analysis pipeline (including GPT-4 Vision) to surface UI-level personalization cues. Key contributions include a working prototype for mobile persona simulation, empirical demonstrations across multiple app categories, and a design for a user-facing sandbox plus a roadmap for extensions like GPT-based persona generation and UI summarization. This approach aims to enhance privacy awareness, enable transparency audits, and empower users to reason about how their digital selves are shaped by contextual data.

Abstract

Mobile applications increasingly rely on sensor data to infer user context and deliver personalized experiences. Yet the mechanisms behind this personalization remain opaque to users and researchers alike. This paper presents a sandbox system that uses sensor spoofing and persona simulation to audit and visualize how mobile apps respond to inferred behaviors. Rather than treating spoofing as adversarial, we demonstrate its use as a tool for behavioral transparency and user empowerment. Our system injects multi-sensor profiles - generated from structured, lifestyle-based personas - into Android devices in real time, enabling users to observe app responses to contexts such as high activity, location shifts, or time-of-day changes. With automated screenshot capture and GPT-4 Vision-based UI summarization, our pipeline helps document subtle personalization cues. Preliminary findings show measurable app adaptations across fitness, e-commerce, and everyday service apps such as weather and navigation. We offer this toolkit as a foundation for privacy-enhancing technologies and user-facing transparency interventions.

Figures (8)

• Figure 1: Simulated persona of Lila Rodriguez. Left: profile image. Right: JSON-style demographic and behavioral traits used to generate sensor spoofing patterns for mobile personalization evaluation.
• Figure 2: Current pipeline for Persona Generation and Real-Time Data Spoofing.
• Figure 3: Simulated high step activity for the "Fitness Enthusiast" persona. Step counter spoofed to emulate frequent walking and trigger fitness tracking behavior.
• Figure 4: Fitness app reacts to spoofed step data by issuing a reward badge, demonstrating behavior change in response to spoofed physical activity.
• Figure 5: GPS spoofed to Piazza del Colosseo, Rome. Location change triggers contextual adjustments across location-aware apps.