MELDAE: A Framework for Micro-Expression Spotting, Detection, and Automatic Evaluation in In-the-Wild Conversational Scenes
Yigui Feng, Qinglin Wang, Yang Liu, Ke Liu, Haotian Mo, Enhao Huang, Gencheng Liu, Mingzhe Liu, Jie Liu
TL;DR
The paper tackles micro-expression analysis in real-world conversational settings, addressing the lack of suitable in-the-wild data and accurate temporal localization. It introduces the WDMD dataset and the MELDAE framework, an encoder-enhancer-decoder architecture with learnable micro-expression tokens and a Boundary-Aware Loss to sharpen onset and offset predictions. The approach achieves state-of-the-art results on WDMD, notably boosting the $F1_{DR}$ localization metric by $17.72\%$ over the strongest baseline, and demonstrates strong generalization to laboratory benchmarks like CAS(ME)$^2$. This work provides a practical foundation for automatic micro-expression analysis in natural conversations and paves the way for real-time, in-the-wild affective computing applications.
Abstract
Accurately analyzing spontaneous, unconscious micro-expressions is crucial for revealing true human emotions, but this task remains challenging in wild scenarios, such as natural conversation. Existing research largely relies on datasets from controlled laboratory environments, and their performance degrades dramatically in the real world. To address this issue, we propose three contributions: the first micro-expression dataset focused on conversational-in-the-wild scenarios; an end-to-end localization and detection framework, MELDAE; and a novel boundary-aware loss function that improves temporal accuracy by penalizing onset and offset errors. Extensive experiments demonstrate that our framework achieves state-of-the-art results on the WDMD dataset, improving the key F1_{DR} localization metric by 17.72% over the strongest baseline, while also demonstrating excellent generalization capabilities on existing benchmarks.