Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spontaneous Speech-Based Suicide Risk Detection Using Whisper and Large Language Models

Ziyun Cui, Chang Lei, Wen Wu, Yinan Duan, Diyang Qu, Ji Wu, Runsen Chen, Chao Zhang

TL;DR

This paper studies the automatic detection of suicide risk based on spontaneous speech from adolescents, and collects a Mandarin dataset with 15 hours of suicide speech from more than a thousand adolescents aged from ten to eighteen for experiments.

Abstract

The early detection of suicide risk is important since it enables the intervention to prevent potential suicide attempts. This paper studies the automatic detection of suicide risk based on spontaneous speech from adolescents, and collects a Mandarin dataset with 15 hours of suicide speech from more than a thousand adolescents aged from ten to eighteen for our experiments. To leverage the diverse acoustic and linguistic features embedded in spontaneous speech, both the Whisper speech model and textual large language models (LLMs) are used for suicide risk detection. Both all-parameter finetuning and parameter-efficient finetuning approaches are used to adapt the pre-trained models for suicide risk detection, and multiple audio-text fusion approaches are evaluated to combine the representations of Whisper and the LLM. The proposed system achieves a detection accuracy of 0.807 and an F1-score of 0.846 on the test set with 119 subjects, indicating promising potential for real suicide risk detection applications.

Spontaneous Speech-Based Suicide Risk Detection Using Whisper and Large Language Models

TL;DR

This paper studies the automatic detection of suicide risk based on spontaneous speech from adolescents, and collects a Mandarin dataset with 15 hours of suicide speech from more than a thousand adolescents aged from ten to eighteen for experiments.

Abstract

The early detection of suicide risk is important since it enables the intervention to prevent potential suicide attempts. This paper studies the automatic detection of suicide risk based on spontaneous speech from adolescents, and collects a Mandarin dataset with 15 hours of suicide speech from more than a thousand adolescents aged from ten to eighteen for our experiments. To leverage the diverse acoustic and linguistic features embedded in spontaneous speech, both the Whisper speech model and textual large language models (LLMs) are used for suicide risk detection. Both all-parameter finetuning and parameter-efficient finetuning approaches are used to adapt the pre-trained models for suicide risk detection, and multiple audio-text fusion approaches are evaluated to combine the representations of Whisper and the LLM. The proposed system achieves a detection accuracy of 0.807 and an F1-score of 0.846 on the test set with 119 subjects, indicating promising potential for real suicide risk detection applications.
Paper Structure (15 sections, 2 figures, 6 tables)

This paper contains 15 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Suicide Data
  3. Automatic Suicide Risk Detection
  4. Speech foundation model: Whisper
  5. Large Language Models
  6. Finetuning strategy
  7. Fusion methods
  8. Experiment Setup
  9. Baselines
  10. Implementation Details
  11. Results and Discussion
  12. Comparison of Different Tuning and Fusion Strategies
  13. Analysis of Prompt Influence when Finetuning
  14. Conclusions
  15. Acknowledgement

Figures (2)

  • Figure 1: Overall pipeline. Modal fusion on acoustic feature and text feature is performed, followed by a classifier for detection of suicide risk.
  • Figure 2: Model structure of two different fusion methods. For concatenation, the speech and text feature are extracted separately and pooled before concatenated. For in-context, speech embedding is mapped to LLM's hidden space and concatenated with embedded tokens, before fed into decoders of LLMs.