Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical Dual-Head Model for Suicide Risk Assessment via MentalRoBERTa

Chang Yang, Ziyi Wang, Wangfeng Tan, Zhiting Tan, Changrui Ji, Zhiming Zhou

TL;DR

The paper tackles suicide risk assessment from social media by addressing severe class imbalance and temporal evolution using a hierarchical dual-head model built on MentalRoBERTa. It combines CORAL ordinal regression with a standard classifier, sharing a sequence representation augmented by explicit time-interval embeddings and a 3-layer Transformer to model inter-post dynamics; training employs a tri-objective loss with fixed weights $0.5$, $0.3$, and $0.2$, along with partial model freezing and mixed-precision training. Data augmentation comprises in-sample LLM-generated variants and external data via LLM-assisted labeling, yielding a more balanced dataset and stronger generalization, validated through 5-fold stratified cross-validation using macro F1 as the primary metric. On augmented data, the model achieves Macro F1 $0.5098$, MAE $0.6474$, and Quadratic Weighted Kappa $0.4692$, outperforming strong baselines and ablations that confirm the importance of temporal modeling and the dual-head design. The findings highlight the practical potential for more reliable, scalable suicide risk screening and intervention planning in real-world social media monitoring pipelines.

Abstract

Social media platforms have become important sources for identifying suicide risk, but automated detection systems face multiple challenges including severe class imbalance, temporal complexity in posting patterns, and the dual nature of risk levels as both ordinal and categorical. This paper proposes a hierarchical dual-head neural network based on MentalRoBERTa for suicide risk classification into four levels: indicator, ideation, behavior, and attempt. The model employs two complementary prediction heads operating on a shared sequence representation: a CORAL (Consistent Rank Logits) head that preserves ordinal relationships between risk levels, and a standard classification head that enables flexible categorical distinctions. A 3-layer Transformer encoder with 8-head multi-head attention models temporal dependencies across post sequences, while explicit time interval embeddings capture posting behavior dynamics. The model is trained with a combined loss function (0.5 CORAL + 0.3 Cross-Entropy + 0.2 Focal Loss) that simultaneously addresses ordinal structure preservation, overconfidence reduction, and class imbalance. To improve computational efficiency, we freeze the first 6 layers (50%) of MentalRoBERTa and employ mixed-precision training. The model is evaluated using 5-fold stratified cross-validation with macro F1 score as the primary metric.

Hierarchical Dual-Head Model for Suicide Risk Assessment via MentalRoBERTa

TL;DR

The paper tackles suicide risk assessment from social media by addressing severe class imbalance and temporal evolution using a hierarchical dual-head model built on MentalRoBERTa. It combines CORAL ordinal regression with a standard classifier, sharing a sequence representation augmented by explicit time-interval embeddings and a 3-layer Transformer to model inter-post dynamics; training employs a tri-objective loss with fixed weights , , and , along with partial model freezing and mixed-precision training. Data augmentation comprises in-sample LLM-generated variants and external data via LLM-assisted labeling, yielding a more balanced dataset and stronger generalization, validated through 5-fold stratified cross-validation using macro F1 as the primary metric. On augmented data, the model achieves Macro F1 , MAE , and Quadratic Weighted Kappa , outperforming strong baselines and ablations that confirm the importance of temporal modeling and the dual-head design. The findings highlight the practical potential for more reliable, scalable suicide risk screening and intervention planning in real-world social media monitoring pipelines.

Abstract

Social media platforms have become important sources for identifying suicide risk, but automated detection systems face multiple challenges including severe class imbalance, temporal complexity in posting patterns, and the dual nature of risk levels as both ordinal and categorical. This paper proposes a hierarchical dual-head neural network based on MentalRoBERTa for suicide risk classification into four levels: indicator, ideation, behavior, and attempt. The model employs two complementary prediction heads operating on a shared sequence representation: a CORAL (Consistent Rank Logits) head that preserves ordinal relationships between risk levels, and a standard classification head that enables flexible categorical distinctions. A 3-layer Transformer encoder with 8-head multi-head attention models temporal dependencies across post sequences, while explicit time interval embeddings capture posting behavior dynamics. The model is trained with a combined loss function (0.5 CORAL + 0.3 Cross-Entropy + 0.2 Focal Loss) that simultaneously addresses ordinal structure preservation, overconfidence reduction, and class imbalance. To improve computational efficiency, we freeze the first 6 layers (50%) of MentalRoBERTa and employ mixed-precision training. The model is evaluated using 5-fold stratified cross-validation with macro F1 score as the primary metric.
Paper Structure (16 sections, 8 equations, 7 figures, 2 tables)

This paper contains 16 sections, 8 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation
  4. Model Architecture
  5. Loss Function
  6. Training Strategy
  7. Data Augmentation
  8. Motivation: Addressing Class Imbalance
  9. Method 1: In-Sample Augmentation via LLM-Generated Variants
  10. Method 2: External Data Acquisition via Crawling and LLM-Assisted Labeling
  11. Experiments
  12. Experimental Setup
  13. Impact of Data Augmentation
  14. Comparison with Baseline Models
  15. Ablation Study
  16. ...and 1 more sections

Figures (7)

  • Figure 1: Overview of the hierarchical dual-head architecture.
  • Figure 2: Original Dataset Distribution by Risk Level
  • Figure 3: Data Augmentation: Overview of combining in-sample LLM-based augmentation and external data acquisition with LLM-assisted labeling.
  • Figure 4: Percentage Growth in Dataset Size per Risk Level After Augmentation
  • Figure 5: Performance comparison between original and augmented datasets.
  • ...and 2 more figures