Emotion Talk: Emotional Support via Audio Messages for Psychological Assistance

TL;DR

Emotion Talk addresses the need for continuous psychological support by analyzing Portuguese audio messages to detect emotions and generate empathetic responses. The system integrates an end-to-end pipeline—audio processing and Mel spectrograms, Whisper transcription, Emotion2Vec+ emotion detection, BERT-based sentiment analysis, GPT-3.5 Turbo for response generation, plus automated report generation and clinician email delivery. On emoUERJ Portuguese data, Emotion2Vec+ achieves about 0.76 accuracy and 0.77 F1, outperforming baselines and supporting real-time emergency responses. This work offers a scalable approach to augment therapy with language-specific emotional support, extendable to clinics and underserved areas, while acknowledging the need for offline capabilities and multilingual expansion.

Abstract

This paper presents "Emotion Talk," a system designed to provide continuous emotional support through audio messages for psychological assistance. The primary objective is to offer consistent support to patients outside traditional therapy sessions by analyzing audio messages to detect emotions and generate appropriate responses. The solution focuses on Portuguese-speaking users, ensuring that the system is linguistically and culturally relevant. This system aims to complement and enhance the psychological follow-up process conducted by therapists, providing immediate and accessible assistance, especially in emergency situations where rapid response is crucial. Experimental results demonstrate the effectiveness of the proposed system, highlighting its potential in applications of psychological support.