Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Facial Biomarkers for Depression through Temporal Analysis of Action Units

Aditya Parikh, Misha Sadeghi, Robert Richer, Lydia Helene Rupp, Lena Schindler-Gmelch, Marie Keinert, Malin Hager, Klara Capito, Farnaz Rahimi, Bernhard Egger, Matthias Berking, Bjoern M. Eskofier

TL;DR

This work investigates objective biomarkers for depression by analyzing temporal dynamics of facial action units (AUs) and associated emotions. Using data from EMPkinS, it extracts AU intensities and emotion expressivity, applies PCA and multiple clustering methods, and evaluates time-series classifiers to distinguish depressed from healthy individuals. The results show elevated sadness-linked AUs (AU1, AU4, AU15) and reduced happiness expressions in depressed participants, with PCA revealing that a small set of components captures most variance and Agglomerative clustering uncovering distinct patterns. The findings support non-invasive, automated facial analysis as a supplemental tool for depression screening and highlight the value of integrating multi-modal data to enhance diagnostic accuracy in real-world settings.

Abstract

Depression is characterized by persistent sadness and loss of interest, significantly impairing daily functioning and now a widespread mental disorder. Traditional diagnostic methods rely on subjective assessments, necessitating objective approaches for accurate diagnosis. Our study investigates the use of facial action units (AUs) and emotions as biomarkers for depression. We analyzed facial expressions from video data of participants classified with or without depression. Our methodology involved detailed feature extraction, mean intensity comparisons of key AUs, and the application of time series classification models. Furthermore, we employed Principal Component Analysis (PCA) and various clustering algorithms to explore the variability in emotional expression patterns. Results indicate significant differences in the intensities of AUs associated with sadness and happiness between the groups, highlighting the potential of facial analysis in depression assessment.

Exploring Facial Biomarkers for Depression through Temporal Analysis of Action Units

TL;DR

This work investigates objective biomarkers for depression by analyzing temporal dynamics of facial action units (AUs) and associated emotions. Using data from EMPkinS, it extracts AU intensities and emotion expressivity, applies PCA and multiple clustering methods, and evaluates time-series classifiers to distinguish depressed from healthy individuals. The results show elevated sadness-linked AUs (AU1, AU4, AU15) and reduced happiness expressions in depressed participants, with PCA revealing that a small set of components captures most variance and Agglomerative clustering uncovering distinct patterns. The findings support non-invasive, automated facial analysis as a supplemental tool for depression screening and highlight the value of integrating multi-modal data to enhance diagnostic accuracy in real-world settings.

Abstract

Depression is characterized by persistent sadness and loss of interest, significantly impairing daily functioning and now a widespread mental disorder. Traditional diagnostic methods rely on subjective assessments, necessitating objective approaches for accurate diagnosis. Our study investigates the use of facial action units (AUs) and emotions as biomarkers for depression. We analyzed facial expressions from video data of participants classified with or without depression. Our methodology involved detailed feature extraction, mean intensity comparisons of key AUs, and the application of time series classification models. Furthermore, we employed Principal Component Analysis (PCA) and various clustering algorithms to explore the variability in emotional expression patterns. Results indicate significant differences in the intensities of AUs associated with sadness and happiness between the groups, highlighting the potential of facial analysis in depression assessment.
Paper Structure (22 sections, 1 equation, 18 figures, 5 tables)

This paper contains 22 sections, 1 equation, 18 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Data Collection
  5. Feature Engineering
  6. Data Preprocessing
  7. Depression-Indicative Action Units
  8. Feature Extraction
  9. Mean Intensity Comparison
  10. PCA Clustering
  11. Silhouette Score Analysis
  12. Time Series Classification Performance
  13. Results and Discussion
  14. Descriptive Statistics
  15. Mean Intensity Comparison of the Dominant Action Units
  16. ...and 7 more sections

Figures (18)

  • Figure 1: Example of AU1: Inner Brow Raiser r7
  • Figure 2: Example of AU6: Cheek Raiser r7
  • Figure 3: Example of AU15: Lip Corner Depressor r7
  • Figure 4: Distribution of participants across three categories in the EmpkinS D02 dataset: Depressed ($n=46$), Healthy ($n=43$), and Sub-clinical ($n=9$).
  • Figure 5: Comparison of Average Intensity of Inner Eyebrow Raise (AU1) Over Time Between Depressed and Non-Depressed Patients.
  • ...and 13 more figures