DIANet: A Phase-Aware Dual-Stream Network for Micro-Expression Recognition via Dynamic Images

TL;DR

MER is challenged by ultra-short, localized facial cues; this paper introduces DIANet, a phase-aware dual-stream network that uses onset–apex and apex–offset dynamic images to capture rising and falling temporal dynamics. Each phase is encoded as a dynamic image ($d_{OA}$, $d_{AO}$) via ARP with weights $\alpha_t = 2t - T - 1$ and $\alpha_t' = T + 1 - 2t$, and features from both streams are fused through a cross-attention mechanism while a cosine-consistency loss $\mathcal{L}_{cons}$ aligns phase representations. Evaluated on CASME-II, SAMM, and MMEW with LOSO, DIANet achieves 70.00%/68.89%/64.24% accuracy respectively, outperforming or matching prior DI-based MER methods without auxiliary modalities. Ablation shows EfficientNetV2 as an effective backbone, and the cross-attention fusion plus phase-wise DI inputs outperform single-phase or holistic DI baselines, underscoring the value of explicit phase modeling. This phase-aware dual-stream approach offers a lightweight, generalizable path for improving MER in both controlled and naturalistic settings, with potential extensions to other motion representations and temporal uncertainty estimation.

Abstract

Micro-expressions are brief, involuntary facial movements that typically last less than half a second and often reveal genuine emotions. Accurately recognizing these subtle expressions is critical for applications in psychology, security, and behavioral analysis. However, micro-expression recognition (MER) remains a challenging task due to the subtle and transient nature of facial cues and the limited availability of annotated data. While dynamic image (DI) representations have been introduced to summarize temporal motion into a single frame, conventional DI-based methods often overlook the distinct characteristics of different temporal phases within a micro-expression. To address this issue, this paper proposes a novel dual-stream framework, DIANet, which leverages phase-aware dynamic images - one encoding the onset-to-apex phase and the other capturing the apex-to-offset phase. Each stream is processed by a dedicated convolutional neural network, and a cross-attention fusion module is employed to adaptively integrate features from both streams based on their contextual relevance. Extensive experiments conducted on three benchmark MER datasets (CASME-II, SAMM, and MMEW) demonstrate that the proposed method consistently outperforms conventional single-phase DI-based approaches. The results highlight the importance of modeling temporal phase information explicitly and suggest a promising direction for advancing MER.

Figures (2)

• Figure 1: The workflow of our proposed dual-stream dynamic imaging for micro-expression recognition
• Figure 2: The cross-attention fusion block in DIANet. Inputs $f1$ and $f2$ are feature vectors obtained from the outputs of two EfficientNetV2 backbones corresponding to the onset–apex and apex–offset streams, respectively. Each is projected into query, key, and value spaces, and mutual attention is computed to produce a fused representation