High Semantic Features for the Continual Learning of Complex Emotions: a Lightweight Solution

TL;DR

This work tackles continual learning for complex emotion recognition by comparing two feature paradigms: CNN-based deep features and high-level Action Units (AUs). It demonstrates that AUs are non-transient, semantically rich cues that, when combined with Gaussian classifiers, yield strong, memory-efficient incremental learning performance on the CFEE dataset, achieving approximately $0.75$ accuracy while maintaining a lightweight footprint. CNN-based features show limited discriminative power for complex emotions and incur higher memory costs, underscoring the advantage of AU-driven approaches for embedded affective computing. Overall, the study contributes a practical, low-footprint solution for evolving emotion recognition tasks and lays groundwork for future discriminative refinements and unsupervised incremental learning.

Abstract

Incremental learning is a complex process due to potential catastrophic forgetting of old tasks when learning new ones. This is mainly due to transient features that do not fit from task to task. In this paper, we focus on complex emotion recognition. First, we learn basic emotions and then, incrementally, like humans, complex emotions. We show that Action Units, describing facial muscle movements, are non-transient, highly semantical features that outperform those extracted by both shallow and deep convolutional neural networks. Thanks to this ability, our approach achieves interesting results when learning incrementally complex, compound emotions with an accuracy of 0.75 on the CFEE dataset and can be favorably compared to state-of-the-art results. Moreover, it results in a lightweight model with a small memory footprint.

Figures (14)

• Figure 1: Basic and compound emotions maiden2023complexfacialexpressionrecognition.
• Figure 2: Facial Action Coding cfee.
• Figure 3: Incremental training of an ensemble of experts (here, number of tasks is T=2).
• Figure 4: Inference of the ensemble of experts (for T=2).
• Figure 5: Training an ensemble of AUs based classifiers (for T=2).