Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"Impressively Scary:" Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media Applications

Jack West, Bengisu Cagiltay, Shirley Zhang, Jingjie Li, Kassem Fawaz, Suman Banerjee

TL;DR

This study investigates how users perceive and react to authentic, on-device AI/ML models used by social media apps, specifically Instagram and TikTok. By reverse-engineering real models and conducting semi-structured interviews (N=21) plus a two-week follow-up, the authors reveal that users generally lack awareness of when and how such models operate, and that exposure can trigger both short-term and long-term behavior changes. The work highlights critical transparency gaps and proposes concrete avenues—install-time transparency, run-time indicators, and user controls—to improve user understanding and trust. The findings underscore the practical importance of designing user-centered transparency mechanisms in mobile apps that process private local data, given varying user preferences and usage patterns.

Abstract

Machine learning models deployed locally on social media applications are used for features, such as face filters which read faces in-real time, and they expose sensitive attributes to the apps. However, the deployment of machine learning models, e.g., when, where, and how they are used, in social media applications is opaque to users. We aim to address this inconsistency and investigate how social media user perceptions and behaviors change once exposed to these models. We conducted user studies (N=21) and found that participants were unaware to both what the models output and when the models were used in Instagram and TikTok, two major social media platforms. In response to being exposed to the models' functionality, we observed long term behavior changes in 8 participants. Our analysis uncovers the challenges and opportunities in providing transparency for machine learning models that interact with local user data.

"Impressively Scary:" Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media Applications

TL;DR

This study investigates how users perceive and react to authentic, on-device AI/ML models used by social media apps, specifically Instagram and TikTok. By reverse-engineering real models and conducting semi-structured interviews (N=21) plus a two-week follow-up, the authors reveal that users generally lack awareness of when and how such models operate, and that exposure can trigger both short-term and long-term behavior changes. The work highlights critical transparency gaps and proposes concrete avenues—install-time transparency, run-time indicators, and user controls—to improve user understanding and trust. The findings underscore the practical importance of designing user-centered transparency mechanisms in mobile apps that process private local data, given varying user preferences and usage patterns.

Abstract

Machine learning models deployed locally on social media applications are used for features, such as face filters which read faces in-real time, and they expose sensitive attributes to the apps. However, the deployment of machine learning models, e.g., when, where, and how they are used, in social media applications is opaque to users. We aim to address this inconsistency and investigate how social media user perceptions and behaviors change once exposed to these models. We conducted user studies (N=21) and found that participants were unaware to both what the models output and when the models were used in Instagram and TikTok, two major social media platforms. In response to being exposed to the models' functionality, we observed long term behavior changes in 8 participants. Our analysis uncovers the challenges and opportunities in providing transparency for machine learning models that interact with local user data.

Paper Structure

This paper contains 59 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Understanding of AI/ML Algorithms
  4. Understanding AI/ML Algorithms in Social Media
  5. Ecological Validity in AI/ML Research In Social Media
  6. AI Explainability and Transparency
  7. AI/ML Privacy
  8. Methods
  9. Participants and Recruitment
  10. Semi-Structured Interview Procedure
  11. Establishing Initial Understandings and Assumptions Towards AI/ML and Social Media
  12. Defining Computer Vision and Clarifying Misconceptions About AI/ML
  13. Asking Participants "When" They Think AI/ML Happens in Social Media
  14. AI/ML Model Demonstration and Participant Interaction
  15. Assessing Short-Term and Long-Term Behavior Changes
  16. ...and 44 more sections

Figures (2)

  • Figure 1: A high-level overview of our semi-structured interview, including example responses. The interview consists of five phases: (Phase 1) We explored participants' past interactions with their social media apps. (Phase 2) We identified participants' current understandings and assumptions towards AI/ML. (Phase 3) We conducted a learning activity, including a high-level example of computer vision, and discussed its capabilities. (Phase 4) We demonstrated the AI/ML models for TikTok and Instagram and observed their immediate reactions. (Phase 5) We surveyed short-term and long-term changes in participant behaviors.
  • Figure 2: Example output that participants would observe during the interview. Instagram displays over 500 different decimals and concepts which are ordered from largest decimal to smallest. If the decimal value is closer to one than it Instagram perceives that concept in the image; if the value is closer to zero the concept is not in the image. TikTok observes age and gender age is an exact estimate and gender seems to be a probability. Gender is a probability called boy_prob when the decimal is closer to one it assumes a male gender identity if its closer to zero it assumes a female gender identity. Note that here we use a image generated by GPT-4o mini gpt4omini for illustrative purposes instead of the input picture from real people in respect of individual privacy.