DFME: A New Benchmark for Dynamic Facial Micro-expression Recognition

TL;DR

The paper introduces DFME, the Dynamic Facial Micro-expressions database, to address the persistent data scarcity in automatic micro-expression recognition. DFME contains 7,526 labeled ME videos from 656 participants across three high-frame-rate settings (200–500 fps) with 24 AU annotations and seven emotion categories, collected under a neutralization paradigm and annotated by multiple experts with high reliability. The authors perform a thorough validation by preprocessing faces, applying 10-fold subject-independent cross-validation, and benchmarking a broad set of MER baselines (3D-CNN, hand-crafted, and deep-learning) as well as AU classification, demonstrating that DFME serves as a strong, scalable benchmark for spatiotemporal MER research. They also analyze emotion–AU associations and provide insights into model performance and data balance, highlighting avenues for future improvements, including multimodal data and self-supervised learning. Overall, DFME stands as a substantial resource that enables more robust MER models and standardized evaluation in real-world, high-frame-rate ME analysis.

Abstract

One of the most important subconscious reactions, micro-expression (ME), is a spontaneous, subtle, and transient facial expression that reveals human beings' genuine emotion. Therefore, automatically recognizing ME (MER) is becoming increasingly crucial in the field of affective computing, providing essential technical support for lie detection, clinical psychological diagnosis, and public safety. However, the ME data scarcity has severely hindered the development of advanced data-driven MER models. Despite the recent efforts by several spontaneous ME databases to alleviate this problem, there is still a lack of sufficient data. Hence, in this paper, we overcome the ME data scarcity problem by collecting and annotating a dynamic spontaneous ME database with the largest current ME data scale called DFME (Dynamic Facial Micro-expressions). Specifically, the DFME database contains 7,526 well-labeled ME videos spanning multiple high frame rates, elicited by 671 participants and annotated by more than 20 professional annotators over three years. Furthermore, we comprehensively verify the created DFME, including using influential spatiotemporal video feature learning models and MER models as baselines, and conduct emotion classification and ME action unit classification experiments. The experimental results demonstrate that the DFME database can facilitate research in automatic MER, and provide a new benchmark for this field. DFME will be published via https://mea-lab-421.github.io.

Figures (5)

• Figure 1: Examples of MaE and ME from the same person with a timeline in seconds, both belong to the "Happiness" emotion category. Noteworthy, the onset frame and the offset frame denote the start and end time of an expression respectively, and the apex frame represents the moment when an expression changes most dramatically. White arrows on the face of the apex frame indicate the general directions of facial movements, and the longer and thicker the arrows, the greater the intensity of facial movements.
• Figure 2: Experimental environment for eliciting MEs
• Figure 3: Representative ME Samples of Seven Discrete Emotion Categories in DFME
• Figure 4: Distribution of ME Samples in DFME. Each column represents the total sample number of an emotion category, and the three pieces colored from light to deep show the proportion of samples in PART A, PART B, and PART C, respectively.
• Figure 5: Confusion matrices of baseline methods including 3D-CNN, hand-crafted MER and deep learning MER methods.