Two-Stage Voting for Robust and Efficient Suicide Risk Detection on Social Media

TL;DR

The paper addresses the challenge of detecting suicidal ideation on social media, particularly implicit expressions, by proposing a two-stage voting architecture that balances efficiency and robustness. Stage 1 uses a fine-tuned BERT model to quickly resolve high-confidence explicit cases, while Stage 2 escalates ambiguous inputs through two pathways: a multi-agent LLM voting ensemble and a feature-based ML ensemble that uses psychologically grounded indicators extracted via prompt-engineered LLMs. A fundamental feature extraction module converts LLM-derived indicators into structured vectors suitable for lightweight classifiers, enabling interpretability and cross-domain generalization. Empirical results on explicit-dominant Reddit and implicit-only DeepSuiMind show near-perfect performance on explicit cases ($=98.0\%$ F1) and very high performance on implicit cases ($=99.7\%$ F1), with cross-domain gaps reduced to under $2\%$ and significant reductions in LLM cost. The framework integrates routing, multi-agent reasoning, and psychologically grounded features to deliver efficient, robust suicide risk detection with clear clinical relevance.

Abstract

Suicide rates have risen worldwide in recent years, underscoring the urgent need for proactive prevention strategies. Social media provides valuable signals, as many at-risk individuals - who often avoid formal help due to stigma - choose instead to share their distress online. Yet detecting implicit suicidal ideation, conveyed indirectly through metaphor, sarcasm, or subtle emotional cues, remains highly challenging. Lightweight models like BERT handle explicit signals but fail on subtle implicit ones, while large language models (LLMs) capture nuance at prohibitive computational cost. To address this gap, we propose a two-stage voting architecture that balances efficiency and robustness. In Stage 1, a lightweight BERT classifier rapidly resolves high-confidence explicit cases. In Stage 2, ambiguous inputs are escalated to either (i) a multi-perspective LLM voting framework to maximize recall on implicit ideation, or (ii) a feature-based ML ensemble guided by psychologically grounded indicators extracted via prompt-engineered LLMs for efficiency and interpretability. To the best of our knowledge, this is among the first works to operationalize LLM-extracted psychological features as structured vectors for suicide risk detection. On two complementary datasets - explicit-dominant Reddit and implicit-only DeepSuiMind - our framework outperforms single-model baselines, achieving 98.0% F1 on explicit cases, 99.7% on implicit ones, and reducing the cross-domain gap below 2%, while significantly lowering LLM cost.

Figures (5)

• Figure 1: Two-stage ensemble architecture for suicide risk detection. Stage 1 applies a BERT classifier with length–confidence routing. Stage 2 resolves ambiguous cases through two alternative pathways: (a) BERT+LLM agent voting and (b) BERT+ML voting.
• Figure 2: Cross-domain performance gaps across Reddit (explicit) and DeepSuiMind (implicit) datasets. Bars show absolute gaps in recall, F1, and their average. Smaller values indicate more robust generalization.
• Figure 3: Stage 1 (Reddit): BERT dominates explicit cases, while fundamental features reach $\sim\!91\%$ and LLMs show high variability.
• Figure 4: Stage 2 (Reddit): Ambiguous cases improve classical ML models ($\sim\!95\%$) and highlight the recall–precision trade-off in LLMs.
• Figure 5: Stage 2 (DeepSuiMind): Fundamental features and GPT-4o-mini bearish achieve $>99\%$, surpassing BERT and GPT-5 variants.