Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting Conversational Mental Manipulation with Intent-Aware Prompting

Jiayuan Ma, Hongbin Na, Zimu Wang, Yining Hua, Yue Liu, Wei Wang, Ling Chen

TL;DR

Detecting covert mental manipulation in conversations is challenging due to subtle tactics. The authors introduce Intent-Aware Prompting (IAP), which enhances large language models' Theory of Mind by performing intent summarization for both speakers and using these summaries to detect manipulation. On the MentalManip dataset, IAP outperforms zero-shot, few-shot, and Chain-of-Thought prompting, achieving notable gains in recall and reductions in false negatives, albeit with some increase in false positives. The work demonstrates that jointly reasoning about interlocutors' intents improves detection performance and provides a practical framework and code for implementation in dialogue-based mental health support systems.

Abstract

Mental manipulation severely undermines mental wellness by covertly and negatively distorting decision-making. While there is an increasing interest in mental health care within the natural language processing community, progress in tackling manipulation remains limited due to the complexity of detecting subtle, covert tactics in conversations. In this paper, we propose Intent-Aware Prompting (IAP), a novel approach for detecting mental manipulations using large language models (LLMs), providing a deeper understanding of manipulative tactics by capturing the underlying intents of participants. Experimental results on the MentalManip dataset demonstrate superior effectiveness of IAP against other advanced prompting strategies. Notably, our approach substantially reduces false negatives, helping detect more instances of mental manipulation with minimal misjudgment of positive cases. The code of this paper is available at https://github.com/Anton-Jiayuan-MA/Manip-IAP.

Detecting Conversational Mental Manipulation with Intent-Aware Prompting

TL;DR

Detecting covert mental manipulation in conversations is challenging due to subtle tactics. The authors introduce Intent-Aware Prompting (IAP), which enhances large language models' Theory of Mind by performing intent summarization for both speakers and using these summaries to detect manipulation. On the MentalManip dataset, IAP outperforms zero-shot, few-shot, and Chain-of-Thought prompting, achieving notable gains in recall and reductions in false negatives, albeit with some increase in false positives. The work demonstrates that jointly reasoning about interlocutors' intents improves detection performance and provides a practical framework and code for implementation in dialogue-based mental health support systems.

Abstract

Mental manipulation severely undermines mental wellness by covertly and negatively distorting decision-making. While there is an increasing interest in mental health care within the natural language processing community, progress in tackling manipulation remains limited due to the complexity of detecting subtle, covert tactics in conversations. In this paper, we propose Intent-Aware Prompting (IAP), a novel approach for detecting mental manipulations using large language models (LLMs), providing a deeper understanding of manipulative tactics by capturing the underlying intents of participants. Experimental results on the MentalManip dataset demonstrate superior effectiveness of IAP against other advanced prompting strategies. Notably, our approach substantially reduces false negatives, helping detect more instances of mental manipulation with minimal misjudgment of positive cases. The code of this paper is available at https://github.com/Anton-Jiayuan-MA/Manip-IAP.

Paper Structure

This paper contains 20 sections, 3 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Observation
  3. Methodology
  4. Intent Summarization
  5. Remark.
  6. Manipulation Detection
  7. Experiments
  8. Experimental Settings
  9. Dataset.
  10. Evaluation Metrics.
  11. Baselines.
  12. Experimental Results
  13. Main Results.
  14. Human Evaluation of Generated Intents.
  15. Related Work
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Comparison of false negatives and false positives in mental manipulation detection using zero-shot and few-shot prompting.
  • Figure 2: Overall framework of Intent-Aware Prompting (IAP) on mental manipulation detection.
  • Figure 3: Examples of various prompting strategies used in the experiment: Zero-shot Prompting, Few-shot Prompting, Zero-shot CoT Prompting, and Intent-Aware Prompting. Each prompt is designed to assess the presence of mental manipulation in dialogues.
  • Figure 4: Examples of intent summarization, illustrating how dialogue between two individuals can be analyzed to extract the underlying intent behind their statements. Each example provides a clear one-sentence summary for both Person1 and Person2, showcasing differing perspectives and emotional undertones within the conversations.