Optical Flow Techniques for Facial Expression Analysis -- a Practical Evaluation Study

Abstract

Optical flow techniques are becoming increasingly performant and robust when estimating motion in a scene, but their performance has yet to be proven in the area of facial expression recognition. In this work, a variety of optical flow approaches are evaluated across multiple facial expression datasets, so as to provide a consistent performance evaluation. The aim of this work is not to propose a new expression recognition technique, but to understand better the adequacy of existing state-of-the art optical flow for encoding facial motion in the context of facial expression recognition. Our evaluations highlight the fact that motion approximation methods used to overcome motion discontinuities have a significant impact when optical flows are used to characterize facial expressions.

Figures (11)

• Figure 1: Comparison of the performance of several optical flow approaches on the MPI-Sintel, a generic synthetic movie dataset (top) for optical flow analysis, and on a set of facial expression datasets (bottom). Such datasets are used due to the lack of optical flow ground-truth data for facial expression analysis. Although the performance tends to be conclusive on MPI-Sintel, this is not the case for facial expression analysis, where a basic approach such as Farnebäck gives the best performance.
• Figure 2: Datasets used to analyze facial expressions from optical flow. The information in bold represents the final data obtained after the standardization process.
• Figure 3: Selection process of the key images according to the intra-face motion.
• Figure 4: Comparison of analysis from raw data, handcrafted and deep learning processes (based on optical flow and used for facial expression recognition).
• Figure 5: Mean AUC obtained from analysis of the raw flow data with TIM2.