You Never Know a Person, You Only Know Their Defenses: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations

TL;DR

This work introduces PsyDefConv, the first publicly released dialogue dataset annotated with DMRS defense levels for help-seeker utterances, bridging clinical theory and natural language data. To support scalable labeling, the authors also present DMRS Co-Pilot, a four-stage pipeline that contextualizes, screens, validates, and synthesizes defense-level inferences as pre-annotations, achieving substantial annotator agreement and meaningful time savings. Extensive zero-shot and fine-tuning experiments across a wide range of large language models reveal substantial headroom, with macro F1 around 30% and a pronounced bias toward high-adaptive defenses, highlighting the need for theory-informed supervision and richer contextual signals. The resource, tooling, and analyses collectively establish a reproducible foundation for studying defensive functioning in language and guiding future models toward more nuanced, clinically aligned interpretations of defense in conversational discourse.

Abstract

Psychological defenses are strategies, often automatic, that people use to manage distress. Rigid or overuse of defenses is negatively linked to mental health and shapes what speakers disclose and how they accept or resist help. However, defenses are complex and difficult to reliably measure, particularly in clinical dialogues. We introduce PsyDefConv, a dialogue corpus with help seeker utterances labeled for defense level, and DMRS Co-Pilot, a four-stage pipeline that provides evidence-based pre-annotations. The corpus contains 200 dialogues and 4709 utterances, including 2336 help seeker turns, with labeling and Cohen's kappa 0.639. In a counterbalanced study, the co-pilot reduced average annotation time by 22.4%. In expert review, it averaged 4.62 for evidence, 4.44 for clinical plausibility, and 4.40 for insight on a seven-point scale. Benchmarks with strong language models in zero-shot and fine-tuning settings demonstrate clear headroom, with the best macro F1-score around 30% and a tendency to overpredict mature defenses. Corpus analyses confirm that mature defenses are most common and reveal emotion-specific deviations. We will release the corpus, annotations, code, and prompts to support research on defensive functioning in language.

Figures (10)

• Figure 1: Distribution of defense levels (x-axis) across different emotions (y-axis). Values indicate the proportion of each defense level within a given emotion.
• Figure 2: Average defense scores over dialogue progress for seeker utterances expressing shame and sadness. Shame shows stronger early defensiveness and greater fluctuation than sadness.
• Figure 3: The four-stage analysis pipeline of DMRS Co-Pilot. The system follows a cascaded design, moving from broad contextualization and screening to deep validation and synthesis, allowing for the strategic use of different models for tasks with varying computational and reasoning requirements.
• Figure 4: F1-score of the models with respect to different levels of psychological defense mechanisms.
• Figure 5: Confusion matrices of Gemini 2.5 Pro and Ministral-8B on the PsyDefConv dataset.