Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Novel Combined Optical Flow Approach for Comprehensive Micro-Expression Recognition

Vu Tram Anh Khuong, Thi Bich Phuong Man, Luu Tu Nguyen, Thanh Ha Le, Thi Duyen Ngo

TL;DR

This paper tackles Micro-Expression Recognition (MER) by addressing the incomplete motion representation in many optical-flow approaches that consider only the onset-to-apex phase. It proposes a Combined Optical Flow (COF), which computes dense optical flow for both onset-to-apex and apex-to-offset phases with the Farneback method, normalizes magnitudes, and sums them to form a unified representation fed to CNN classifiers such as VGG19 and ResNet50, with training conducted using Adam at a learning rate of $10^{-4}$. In experiments on CASME II and SAMM, COF consistently outperforms single optical-flow baselines, achieving notable gains (e.g., CASME II: 67.35% with VGG19 vs 61.22%; SAMM: 59.26% with VGG19 vs 50.00%). The results demonstrate that apex-to-offset dynamics provide complementary information for accurate micro-expression discrimination, supporting the integration of COF into robust MER pipelines and paving the way for deeper architectures and real-time applications.

Abstract

Facial micro-expressions are brief, involuntary facial movements that reveal hidden emotions. Most Micro-Expression Recognition (MER) methods that rely on optical flow typically focus on the onset-to-apex phase, neglecting the apex-to-offset phase, which holds key temporal dynamics. This study introduces a Combined Optical Flow (COF), integrating both phases to enhance feature representation. COF provides a more comprehensive motion analysis, improving MER performance. Experimental results on CASMEII and SAMM datasets show that COF outperforms single optical flow-based methods, demonstrating its effectiveness in capturing micro-expression dynamics.

A Novel Combined Optical Flow Approach for Comprehensive Micro-Expression Recognition

TL;DR

This paper tackles Micro-Expression Recognition (MER) by addressing the incomplete motion representation in many optical-flow approaches that consider only the onset-to-apex phase. It proposes a Combined Optical Flow (COF), which computes dense optical flow for both onset-to-apex and apex-to-offset phases with the Farneback method, normalizes magnitudes, and sums them to form a unified representation fed to CNN classifiers such as VGG19 and ResNet50, with training conducted using Adam at a learning rate of . In experiments on CASME II and SAMM, COF consistently outperforms single optical-flow baselines, achieving notable gains (e.g., CASME II: 67.35% with VGG19 vs 61.22%; SAMM: 59.26% with VGG19 vs 50.00%). The results demonstrate that apex-to-offset dynamics provide complementary information for accurate micro-expression discrimination, supporting the integration of COF into robust MER pipelines and paving the way for deeper architectures and real-time applications.

Abstract

Facial micro-expressions are brief, involuntary facial movements that reveal hidden emotions. Most Micro-Expression Recognition (MER) methods that rely on optical flow typically focus on the onset-to-apex phase, neglecting the apex-to-offset phase, which holds key temporal dynamics. This study introduces a Combined Optical Flow (COF), integrating both phases to enhance feature representation. COF provides a more comprehensive motion analysis, improving MER performance. Experimental results on CASMEII and SAMM datasets show that COF outperforms single optical flow-based methods, demonstrating its effectiveness in capturing micro-expression dynamics.
Paper Structure (12 sections, 3 figures, 1 table, 1 algorithm)

This paper contains 12 sections, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Proposed method
  3. Optical Flow Calculation
  4. Optical Flow Combination
  5. Classification
  6. Experiment and Results
  7. Experiment
  8. Dataset
  9. Evaluation Metrics
  10. Implementation Details
  11. Results
  12. Conclusion

Figures (3)

  • Figure 1: Pipeline for MER with Combined Optical Flow
  • Figure 2: Visualization of Onset frame, Apex frame, Offset frame, Optical Flow (Onset-Apex), and COF from CASME-II (top row) and SAMM (bottom row) datasets. (Best viewed in color)
  • Figure 3: Confusion Matrices for Optical Flow and Combined Optical Flow in MER