Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI

Isadora Krsek, Anubha Kabra, Yao Dou, Tarek Naous, Laura A. Dabbish, Alan Ritter, Wei Xu, Sauvik Das

TL;DR

This study evaluates how AI-assisted, span-level self-disclosure detection can aid users in balancing the benefits and privacy risks of posting on pseudonymous platforms like Reddit. Using a state-of-the-art 19-category and a binary disclosure detector as a technology probe, the authors conduct 75-minute interviews with 21 Reddit users, examining how model outputs affect risk awareness and posting decisions. They find that while the tool is imperfect, users value granular category labels for explanation and emphasize the need to account for posting context, norms, and individual threat models. The work proposes design directions—high-confidence surfacing, fine-grained explanations, de-risking rephrasings, and context-aware outputs—that could guide practical deployment of AI-powered privacy decision aids in social platforms. Overall, the paper highlights the social-technical considerations essential for usable, user-centered AI tools that support privacy in online self-disclosures.

Abstract

In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users (e.g., I can vent about my in-laws to understanding strangers), but the privacy risks are more abstract (e.g., will my partner be able to tell that this is me?). Prior work has sought to develop natural language processing (NLP) tools that help users identify potentially risky self-disclosures in their text, but none have been designed for or evaluated with the users they hope to protect. Absent this assessment, these tools will be limited by the social-technical gap: users need assistive tools that help them make informed decisions, not paternalistic tools that tell them to avoid self-disclosure altogether. To bridge this gap, we conducted a study with N = 21 Reddit users; we had them use a state-of-the-art NLP disclosure detection model on two of their authored posts and asked them questions to understand if and how the model helped, where it fell short, and how it could be improved to help them make more informed decisions. Despite its imperfections, users responded positively to the model and highlighted its use as a tool that can help them catch mistakes, inform them of risks they were unaware of, and encourage self-reflection. However, our work also shows how, to be useful and usable, AI for supporting privacy decision-making must account for posting context, disclosure norms, and users' lived threat models, and provide explanations that help contextualize detected risks.

Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI

TL;DR

This study evaluates how AI-assisted, span-level self-disclosure detection can aid users in balancing the benefits and privacy risks of posting on pseudonymous platforms like Reddit. Using a state-of-the-art 19-category and a binary disclosure detector as a technology probe, the authors conduct 75-minute interviews with 21 Reddit users, examining how model outputs affect risk awareness and posting decisions. They find that while the tool is imperfect, users value granular category labels for explanation and emphasize the need to account for posting context, norms, and individual threat models. The work proposes design directions—high-confidence surfacing, fine-grained explanations, de-risking rephrasings, and context-aware outputs—that could guide practical deployment of AI-powered privacy decision aids in social platforms. Overall, the paper highlights the social-technical considerations essential for usable, user-centered AI tools that support privacy in online self-disclosures.

Abstract

In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users (e.g., I can vent about my in-laws to understanding strangers), but the privacy risks are more abstract (e.g., will my partner be able to tell that this is me?). Prior work has sought to develop natural language processing (NLP) tools that help users identify potentially risky self-disclosures in their text, but none have been designed for or evaluated with the users they hope to protect. Absent this assessment, these tools will be limited by the social-technical gap: users need assistive tools that help them make informed decisions, not paternalistic tools that tell them to avoid self-disclosure altogether. To bridge this gap, we conducted a study with N = 21 Reddit users; we had them use a state-of-the-art NLP disclosure detection model on two of their authored posts and asked them questions to understand if and how the model helped, where it fell short, and how it could be improved to help them make more informed decisions. Despite its imperfections, users responded positively to the model and highlighted its use as a tool that can help them catch mistakes, inform them of risks they were unaware of, and encourage self-reflection. However, our work also shows how, to be useful and usable, AI for supporting privacy decision-making must account for posting context, disclosure norms, and users' lived threat models, and provide explanations that help contextualize detected risks.

Paper Structure

This paper contains 42 sections, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Tension Between the Need for Self-Disclosure and the Risks
  4. Past Approaches to Alleviating the Risks of Self-Disclosure
  5. Non AI-based Tools
  6. AI-based Tools/existing NLP Approaches to detecting self-disclosure
  7. The Need for User Engagement in the Design of AI Technology
  8. Self-Disclosure Detection Model
  9. Methodology
  10. Recruitment
  11. Pre-study Survey
  12. Interview
  13. User Assessment of Model Output on Their Posts
  14. Data Analysis
  15. Results
  16. ...and 27 more sections

Figures (5)

  • Figure 1: AI-powered, span-level self-disclosure detection helps users in reviewing their posts to improve privacy.
  • Figure 2: The model classifies each word to a label.
  • Figure 3: Image depicting how the disclosure detection spans of both versions of the model were presented to participants. On the left is the binary disclosure version, and on the right is the categorical disclosure version.
  • Figure : INTERVIEW METHODOLGY FLOWCHART
  • Figure : BARPLOT OF ACCEPTED VS. REJECTED DISCLOSURE DETECTION SPANS, BY MODEL TYPE