Assessing ML Classification Algorithms and NLP Techniques for Depression Detection: An Experimental Case Study

TL;DR

This study evaluates machine learning and NLP approaches for depression detection using the DAIC-WOZ dataset, focusing on data cleaning, feature selection, and model choice across RF, XGBoost, and SVM. The authors demonstrate that Random Forest and XGBoost can reach about 84% accuracy, outperforming SVM and prior literature on the same data, with sentiment and timing features playing nuanced roles. They also highlight challenges from small, imbalanced samples with PTSD comorbidity and call for broader model testing, richer features, and more balanced datasets. The findings suggest that careful feature engineering and robust classifier selection can enable more scalable, automated screening for depression in mental health contexts where expert resources are limited.

Abstract

Depression has affected millions of people worldwide and has become one of the most common mental disorders. Early mental disorder detection can reduce costs for public health agencies and prevent other major comorbidities. Additionally, the shortage of specialized personnel is very concerning since Depression diagnosis is highly dependent on expert professionals and is time-consuming. Recent research has evidenced that machine learning (ML) and Natural Language Processing (NLP) tools and techniques have significantly bene ted the diagnosis of depression. However, there are still several challenges in the assessment of depression detection approaches in which other conditions such as post-traumatic stress disorder (PTSD) are present. These challenges include assessing alternatives in terms of data cleaning and pre-processing techniques, feature selection, and appropriate ML classification algorithms. This paper tackels such an assessment based on a case study that compares different ML classifiers, specifically in terms of data cleaning and pre-processing, feature selection, parameter setting, and model choices. The case study is based on the Distress Analysis Interview Corpus - Wizard-of-Oz (DAIC-WOZ) dataset, which is designed to support the diagnosis of mental disorders such as depression, anxiety, and PTSD. Besides the assessment of alternative techniques, we were able to build models with accuracy levels around 84% with Random Forest and XGBoost models, which is significantly higher than the results from the comparable literature which presented the level of accuracy of 72% from the SVM model.

Figures (1)

• Figure 1: Workflow Overview to select the optimum model. The red arrow illustrates that model training may loop back to estimator selection scikit-learn_settings, pre-processing, feature engineering, and parameter setting.