Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Behind the Smile: Mental Health Implications of Mother-Infant Interactions Revealed Through Smile Analysis

Adi Dust, Pat Levitt, Maja Matarić

TL;DR

This paper addresses whether maternal mental health can be inferred from time-resolved maternal smiles during 6–12 month infant play. It leverages OpenFace to extract facial features from 94 mother–infant videos and trains temporal-windowed neural networks to predict self-reported mental-health scores (PHQ9, ACES, PSS, PEARLS). The study identifies key smile dynamics—onset amplitude, onset duration, offset duration, and total duration—that correlate with maternal distress, with window sizes of 3–5 smiles yielding the best predictive performance (MAE, RMSE) and correlations that strengthen over the interaction. Developmental stage modulates these effects, showing distinct smile patterns between 6 and 12 months, and the work demonstrates the potential of noninvasive, temporally-resolved smile analysis to gatekeep maternal mental health insights in caregiving contexts, while highlighting the need for multimodal, longer, and more diverse studies.

Abstract

Mothers of infants have specific demands in fostering emotional bonds with their children, characterized by dynamics that are different from adult-adult interactions, notably requiring heightened maternal emotional regulation. In this study, we analyzed maternal emotional state by modeling maternal emotion regulation reflected in smiles. The dataset comprises N=94 videos of approximately 3 plus or minus 1-minutes, capturing free play interactions between 6 and 12-month-old infants and their mothers. Corresponding demographic details of self-reported maternal mental health provide variables for determining mothers' relations to emotions measured during free play. In this work, we employ diverse methodological approaches to explore the temporal evolution of maternal smiles. Our findings reveal a correlation between the temporal dynamics of mothers' smiles and their emotional state. Furthermore, we identify specific smile features that correlate with maternal emotional state, thereby enabling informed inferences with existing literature on general smile analysis. This study offers insights into emotional labor, defined as the management of one's own emotions for the benefit of others, and emotion regulation entailed in mother-infant interactions.

Behind the Smile: Mental Health Implications of Mother-Infant Interactions Revealed Through Smile Analysis

TL;DR

This paper addresses whether maternal mental health can be inferred from time-resolved maternal smiles during 6–12 month infant play. It leverages OpenFace to extract facial features from 94 mother–infant videos and trains temporal-windowed neural networks to predict self-reported mental-health scores (PHQ9, ACES, PSS, PEARLS). The study identifies key smile dynamics—onset amplitude, onset duration, offset duration, and total duration—that correlate with maternal distress, with window sizes of 3–5 smiles yielding the best predictive performance (MAE, RMSE) and correlations that strengthen over the interaction. Developmental stage modulates these effects, showing distinct smile patterns between 6 and 12 months, and the work demonstrates the potential of noninvasive, temporally-resolved smile analysis to gatekeep maternal mental health insights in caregiving contexts, while highlighting the need for multimodal, longer, and more diverse studies.

Abstract

Mothers of infants have specific demands in fostering emotional bonds with their children, characterized by dynamics that are different from adult-adult interactions, notably requiring heightened maternal emotional regulation. In this study, we analyzed maternal emotional state by modeling maternal emotion regulation reflected in smiles. The dataset comprises N=94 videos of approximately 3 plus or minus 1-minutes, capturing free play interactions between 6 and 12-month-old infants and their mothers. Corresponding demographic details of self-reported maternal mental health provide variables for determining mothers' relations to emotions measured during free play. In this work, we employ diverse methodological approaches to explore the temporal evolution of maternal smiles. Our findings reveal a correlation between the temporal dynamics of mothers' smiles and their emotional state. Furthermore, we identify specific smile features that correlate with maternal emotional state, thereby enabling informed inferences with existing literature on general smile analysis. This study offers insights into emotional labor, defined as the management of one's own emotions for the benefit of others, and emotion regulation entailed in mother-infant interactions.
Paper Structure (15 sections, 2 figures, 4 tables)

This paper contains 15 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Maternal Mental Health
  4. Smile Analysis
  5. Mother-Infant Interaction
  6. Methods
  7. Dataset
  8. Maternal Smile Feature Extraction
  9. Temporal Feature Windowing
  10. Model Architecture
  11. Results
  12. Temporal Analysis
  13. Feature Correlation and Developmental Stage
  14. Discussion
  15. Conclusion

Figures (2)

  • Figure 1: A graph of an example smile from the mother in dyad 1. The graph displays the smile onset, apex, and offset.
  • Figure 2: The change in smile feature correlations by emotional state scale scores over each smile in the interaction. The most linearly correlated smile features were selected.