Ada-DF: An Adaptive Label Distribution Fusion Network For Facial Expression Recognition

TL;DR

This work tackles annotation ambiguity in facial expression recognition by adopting a label distribution learning framework. It introduces Ada-DF, a dual-branch network with a label-distribution extracting auxiliary branch and a recognition-focused target branch, fused adaptively via attention to provide richer supervision. Class distribution mining complements label distributions by deriving emotion-level distributions, and a joint multi-task loss guides learning with ramped emphasis on auxiliary versus target tasks. Empirically, Ada-DF achieves state-of-the-art results on RAF-DB, AffectNet, and SFEW, demonstrates robustness to synthetic label noise, and offers interpretable visualizations showing more discriminative features and distributions, highlighting practical impact for real-world FER under uncertainty.

Abstract

Facial expression recognition (FER) plays a significant role in our daily life. However, annotation ambiguity in the datasets could greatly hinder the performance. In this paper, we address FER task via label distribution learning paradigm, and develop a dual-branch Adaptive Distribution Fusion (Ada-DF) framework. One auxiliary branch is constructed to obtain the label distributions of samples. The class distributions of emotions are then computed through the label distributions of each emotion. Finally, those two distributions are adaptively fused according to the attention weights to train the target branch. Extensive experiments are conducted on three real-world datasets, RAF-DB, AffectNet and SFEW, where our Ada-DF shows advantages over the state-of-the-art works.

Figures (6)

• Figure 1: Annotations by 50 volunteers on an image from RAF-DB. The face is annotated with 7 basic emotions, including surprise, fear, disgust, happiness, sadness, anger and neutral. Another two faces are chosen to show the inter-class similarity and intra-class variation.
• Figure 2: Ada-DF contains an auxiliary branch for label distribution generation, and a target branch for final emotion prediction. The attention weights of two attention modules are normalized to integrate the label distributions and the class distributions. The final fused distributions are used to train the target branch via label distribution learning.
• Figure 3: T-SNE visualization of the outputs in the final hidden layer of the baseline and our approach
• Figure 4: Class distributions of 7 basic emotions mined in RAF-DB. The sum of each distribution may not be equal to 1 due to rounding.
• Figure 5: An example of adaptive distribution fusion. The sum of each distribution may not be equal to 1 due to rounding.