Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Secret Use of Large Language Model (LLM)

Zhiping Zhang, Chenxinran Shen, Bingsheng Yao, Dakuo Wang, Tianshi Li

TL;DR

The paper examines why end-users hide their use of LLMs, integrating a qualitative online survey (n=125 secret-use cases) and a controlled experiment (n=300) to reveal that task type primarily drives secret-use intentions through perceived external judgment. It identifies internal and external judgments as core drivers, with external judgments mediating the effect of task context on both passive non-disclosure and active concealment. The work highlights the prevalence of secret LLM usage in critical domains like academic and work tasks, discusses privacy–transparency tensions, and suggests norm-based and policy-informed interventions to promote disclosure. Overall, the study advances understanding of AI transparency from the user perspective and offers practical guidance for designing strategies to encourage responsible and transparent AI use while respecting individual privacy.

Abstract

The advancements of Large Language Models (LLMs) have decentralized the responsibility for the transparency of AI usage. Specifically, LLM users are now encouraged or required to disclose the use of LLM-generated content for varied types of real-world tasks. However, an emerging phenomenon, users' secret use of LLM, raises challenges in ensuring end users adhere to the transparency requirement. Our study used mixed-methods with an exploratory survey (125 real-world secret use cases reported) and a controlled experiment among 300 users to investigate the contexts and causes behind the secret use of LLMs. We found that such secretive behavior is often triggered by certain tasks, transcending demographic and personality differences among users. Task types were found to affect users' intentions to use secretive behavior, primarily through influencing perceived external judgment regarding LLM usage. Our results yield important insights for future work on designing interventions to encourage more transparent disclosure of the use of LLMs or other AI technologies.

Secret Use of Large Language Model (LLM)

TL;DR

The paper examines why end-users hide their use of LLMs, integrating a qualitative online survey (n=125 secret-use cases) and a controlled experiment (n=300) to reveal that task type primarily drives secret-use intentions through perceived external judgment. It identifies internal and external judgments as core drivers, with external judgments mediating the effect of task context on both passive non-disclosure and active concealment. The work highlights the prevalence of secret LLM usage in critical domains like academic and work tasks, discusses privacy–transparency tensions, and suggests norm-based and policy-informed interventions to promote disclosure. Overall, the study advances understanding of AI transparency from the user perspective and offers practical guidance for designing strategies to encourage responsible and transparent AI use while respecting individual privacy.

Abstract

The advancements of Large Language Models (LLMs) have decentralized the responsibility for the transparency of AI usage. Specifically, LLM users are now encouraged or required to disclose the use of LLM-generated content for varied types of real-world tasks. However, an emerging phenomenon, users' secret use of LLM, raises challenges in ensuring end users adhere to the transparency requirement. Our study used mixed-methods with an exploratory survey (125 real-world secret use cases reported) and a controlled experiment among 300 users to investigate the contexts and causes behind the secret use of LLMs. We found that such secretive behavior is often triggered by certain tasks, transcending demographic and personality differences among users. Task types were found to affect users' intentions to use secretive behavior, primarily through influencing perceived external judgment regarding LLM usage. Our results yield important insights for future work on designing interventions to encourage more transparent disclosure of the use of LLMs or other AI technologies.
Paper Structure (64 sections, 3 figures, 4 tables)

This paper contains 64 sections, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Secretive Behavior
  4. AI Transparency
  5. Existing Community-Level Requirements of Transparency for LLM Use
  6. Universities
  7. Journalism, Social Media and Blog
  8. Academia
  9. Intergovernmental and governmental organization
  10. Workplaces
  11. Study 1: Online Survey
  12. Survey Design
  13. Qualitative Analysis Methods
  14. Reason of Secret Use Analysis
  15. Scenario of Secret Use Analysis
  16. ...and 49 more sections

Figures (3)

  • Figure 1: Fill-in-the-blank questions for participants to share their experiences of secret use of LLMs
  • Figure 2: Estimates of the percentage of people who at least somewhat agree with the statements related to the internal judgement and the perceived external judgement on the use of LLMs. "GIS"(General Information Search) is the reference condition.
  • Figure 3: Structural equation model showing the mediation analysis of task type, two types of perception on the use of LLMs and two forms of users' intentions to conceal the usage. Values represent standardized regression coefficients. It should be noted that both "Perceived external judgement" and "Internal judgement" in our study are all presented on negative side. Here, CFI: 0.935, SRMR: 0.042, RMSEA: 0.126, p<0.001: ***