Face-GPS: A Comprehensive Technique for Quantifying Facial Muscle Dynamics in Videos
Juni Kim, Zhikang Dong, Pawel Polak
TL;DR
Face-GPS targets the challenge of quantifying facial muscle dynamics from widely available video by creating canonical face representations that are robust to head motion, then measuring muscle activity with DISC-inspired optical flow. The approach couples spectral and wavelet smoothing with a novel Multiple Kernel Smoothing (MKS) framework that fuses multiple facial descriptors via Gaussian radial basis kernels, weighted by a FAN based expression model, yielding smoothed displacement signals $r''_j$ for each facial region. Empirically, the method achieves strong emotion-related muscle movement identification on CK+ and delivers competitive classification performance (85% average accuracy, 86.1% with FAN) using only muscle displacements, underscoring interpretability and potential clinical utility. The work offers a non-invasive, explainable alternative to deep models with applications in remote diagnosis, plastic surgery planning, and security, while acknowledging challenges in accurately modeling boundary regions of the facial manifold.
Abstract
We introduce a novel method that combines differential geometry, kernels smoothing, and spectral analysis to quantify facial muscle activity from widely accessible video recordings, such as those captured on personal smartphones. Our approach emphasizes practicality and accessibility. It has significant potential for applications in national security and plastic surgery. Additionally, it offers remote diagnosis and monitoring for medical conditions such as stroke, Bell's palsy, and acoustic neuroma. Moreover, it is adept at detecting and classifying emotions, from the overt to the subtle. The proposed face muscle analysis technique is an explainable alternative to deep learning methods and a non-invasive substitute to facial electromyography (fEMG).