Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MoodCam: Mood Prediction Through Smartphone-Based Facial Affect Analysis in Real-World Settings

Rahul Islam, Tongze Zhang, Sang Won Bae

TL;DR

MoodCam advances real-world mood assessment by leveraging front-facing smartphone cameras to extract facial-affect primitives and map them to the Circumplex Model of Affect (valence and arousal). The authors collect 15,995 facial-behavior sessions from 25 participants over four weeks using the FacePsy pipeline, and implement three prediction modes: momentary, daily average, and next-day average, achieving AUC values in the 0.58–0.64 (valence) and 0.60–0.63 (arousal) ranges. Feature selection via Random Forest and a LOPO-validated Decision-Tree classifier, complemented by SMOTE for balance, yields robust performance with insights from an ablation study highlighting Eye Area Ratios and Action Units as key signals. The work demonstrates the feasibility of on-device, real-world mood forecasting to support timely interventions and resource planning in mental health, paving the way for unobtrusive, continuous mood monitoring integrated into digital health ecosystems.

Abstract

MoodCam introduces a novel method for assessing mood by utilizing facial affect analysis through the front-facing camera of smartphones during everyday activities. We collected facial behavior primitives during 15,995 real-world phone interactions involving 25 participants over four weeks. We developed three models for timely intervention: momentary, daily average, and next day average. Notably, our models exhibit AUC scores ranging from 0.58 to 0.64 for Valence and 0.60 to 0.63 for Arousal. These scores are comparable to or better than those from some previous studies. This predictive ability suggests that MoodCam can effectively forecast mood trends, providing valuable insights for timely interventions and resource planning in mental health management. The results are promising as they demonstrate the viability of using real-time and predictive mood analysis to aid in mental health interventions and potentially offer preemptive support during critical periods identified through mood trend shifts.

MoodCam: Mood Prediction Through Smartphone-Based Facial Affect Analysis in Real-World Settings

TL;DR

MoodCam advances real-world mood assessment by leveraging front-facing smartphone cameras to extract facial-affect primitives and map them to the Circumplex Model of Affect (valence and arousal). The authors collect 15,995 facial-behavior sessions from 25 participants over four weeks using the FacePsy pipeline, and implement three prediction modes: momentary, daily average, and next-day average, achieving AUC values in the 0.58–0.64 (valence) and 0.60–0.63 (arousal) ranges. Feature selection via Random Forest and a LOPO-validated Decision-Tree classifier, complemented by SMOTE for balance, yields robust performance with insights from an ablation study highlighting Eye Area Ratios and Action Units as key signals. The work demonstrates the feasibility of on-device, real-world mood forecasting to support timely interventions and resource planning in mental health, paving the way for unobtrusive, continuous mood monitoring integrated into digital health ecosystems.

Abstract

MoodCam introduces a novel method for assessing mood by utilizing facial affect analysis through the front-facing camera of smartphones during everyday activities. We collected facial behavior primitives during 15,995 real-world phone interactions involving 25 participants over four weeks. We developed three models for timely intervention: momentary, daily average, and next day average. Notably, our models exhibit AUC scores ranging from 0.58 to 0.64 for Valence and 0.60 to 0.63 for Arousal. These scores are comparable to or better than those from some previous studies. This predictive ability suggests that MoodCam can effectively forecast mood trends, providing valuable insights for timely interventions and resource planning in mental health management. The results are promising as they demonstrate the viability of using real-time and predictive mood analysis to aid in mental health interventions and potentially offer preemptive support during critical periods identified through mood trend shifts.

Paper Structure

This paper contains 29 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Understanding Mood and Evaluation Metrics
  4. Mood Inference and Detection Using Smartphones and Wearables in-the-Wild
  5. User Study and Field Data Collection
  6. Research Design and Participant Demographics
  7. Protocol
  8. Participants
  9. Field Study
  10. Data Collection
  11. Dataset
  12. The Design of MoodCam Models
  13. MoodCam Output
  14. At moment
  15. Daily Average
  16. ...and 14 more sections

Figures (1)

  • Figure 1: Proposed MoodCam system and models, analyzing facial affect data from real-world settings