Evidence-Driven Marker Extraction for Social Media Suicide Risk Detection

TL;DR

The paper tackles the need for interpretable and efficient suicide risk detection from social media by introducing ED-LLM, a multi-task framework based on Mistral-7B that jointly performs clinical marker span extraction and risk classification. It explicitly outputs evidence spans via BIO tagging and optimizes a combined loss $L_{total}$ to align markers with risk predictions, achieving strong marker extraction and competitive risk accuracy on CLPsych benchmarks. Empirical results show ED-LLM outperforms baselines in marker span identification and maintains solid performance in risk classification, with qualitative analysis demonstrating interpretable evidence. The work advances clinically relevant AI by combining evidence extraction with risk assessment, achieving a favorable balance between performance, interpretability, and computational efficiency, and outlining future directions toward robustness and multimodal integration.

Abstract

Early detection of suicide risk from social media text is crucial for timely intervention. While Large Language Models (LLMs) offer promising capabilities in this domain, challenges remain in terms of interpretability and computational efficiency. This paper introduces Evidence-Driven LLM (ED-LLM), a novel approach for clinical marker extraction and suicide risk classification. ED-LLM employs a multi-task learning framework, jointly training a Mistral-7B based model to identify clinical marker spans and classify suicide risk levels. This evidence-driven strategy enhances interpretability by explicitly highlighting textual evidence supporting risk assessments. Evaluated on the CLPsych datasets, ED-LLM demonstrates competitive performance in risk classification and superior capability in clinical marker span identification compared to baselines including fine-tuned LLMs, traditional machine learning, and prompt-based methods. The results highlight the effectiveness of multi-task learning for interpretable and efficient LLM-based suicide risk assessment, paving the way for clinically relevant applications.