Infused Suppression Of Magnification Artefacts For Micro-AU Detection

TL;DR

The paper tackles magnification artefacts in micro-expression AU detection by introducing InfuseNet, a two-branch framework that combines magnified motion features with optical-flow context. A key innovation is successive layer-wise feature infusion from an optical-flow branch into the magnified-motion branch, along with using latent magnified features to avoid reconstruction artefacts. Extensive experiments on six micro-expression databases under the CD6ME protocol show state-of-the-art AU detection performance and robust improvements across small and large AUs, supported by ablations and AU visualization. The work demonstrates that artefact-aware, flow-guided infusion enhances cross-database micro-AU detection, with implications for more reliable fine-grained facial motion analysis in real-world settings.

Abstract

Facial micro-expressions are spontaneous, brief and subtle facial motions that unveil the underlying, suppressed emotions. Detecting Action Units (AUs) in micro-expressions is crucial because it yields a finer representation of facial motions than categorical emotions, effectively resolving the ambiguity among different expressions. One of the difficulties in micro-expression analysis is that facial motions are subtle and brief, thereby increasing the difficulty in correlating facial motion features to AU occurrence. To bridge the subtlety issue, flow-related features and motion magnification are a few common approaches as they can yield descriptive motion changes and increased motion amplitude respectively. While motion magnification can amplify the motion changes, it also accounts for illumination changes and projection errors during the amplification process, thereby creating motion artefacts that confuse the model to learn inauthentic magnified motion features. The problem is further aggravated in the context of a more complicated task where more AU classes are analyzed in cross-database settings. To address this issue, we propose InfuseNet, a layer-wise unitary feature infusion framework that leverages motion context to constrain the Action Unit (AU) learning within an informative facial movement region, thereby alleviating the influence of magnification artefacts. On top of that, we propose leveraging magnified latent features instead of reconstructing magnified samples to limit the distortion and artefacts caused by the projection inaccuracy in the motion reconstruction process. Via alleviating the magnification artefacts, InfuseNet has surpassed the state-of-the-art results in the CD6ME protocol. Further quantitative studies have also demonstrated the efficacy of motion artefacts alleviation.

Figures (5)

• Figure 1: The illustrations of magnification artefacts and feature infusion. Part (A) demonstrates the magnification artefacts generated during the motion amplification, where faces appear deformed. Part (B) presents the proposed feature infusion process, which extracts latent magnified features while counterbalancing the amplified noise using optical flow images.
• Figure 2: Our proposed framework, InfuseNet, alleviates the magnification artefacts via infusing motion context from FrameFlow. Given onset, apex and pseudo-apex frames, we extract the magnified latent features as the input to the FrameMag network. Meanwhile, for FrameFlow, we extract the optical flow and strain features from the onset to apex frames. Following that, we perform successive layer feature infusion from FrameFlow to FrameMag, and then FrameMag predicts the eventual one-hot AUs of the samples.
• Figure 3: The confusion matrix of SSSNet(Left) and InfuseNet-Res18(Right). In comparison, InfuseNet-Res18 yields a more balanced performance across AU classes as compared to SSSNet.
• Figure 4: The comparison of magnification factors. This side experiment identifies the optimal magnification factor for our proposed framework.
• Figure 5: Visualization of the activations to simulate how infusion can eliminate the artefacts from FrameMag. In this sample, we can observe that the infusion helps to pinpoint the precise AU location without sacrificing the features from magnification.