Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual Stream Independence Decoupling for True Emotion Recognition under Masked Expressions

Jinsheng Wei, Xiguang Zhang, Zheng Shi, Guanming Lu

Abstract

Recongnizing true emotions from masked expressions is extremely challenging due to deliberate concealment. Existing paradigms recognize true emotions from masked-expression clips that contain onsetframes just starting to disguise. However, this paradigm may not reflect the actual disguised state, as the onsetframe leaks the true emotional information without reaching a stable disguise state. Thus, this paper introduces a novel apexframe-based paradigm that classifies true emotions from the apexframe with a stable disguised state. Furthermore, this paper proposes a novel dual stream independence decoupling framework that decouples true and disguised emotion features, avoiding the interference of disguised emotions on true emotions. For efficient decoupling, we design a decoupling loss group, comprising two classification losses that learn true emotion and disguised expression features, respectively, and a Hilbert-Schmidt Independence loss that enhances the independence of two features. Experiments demonstrate that the apexframe-based paradigm is challenging, and the proposed decouple framework improves recogntion performances.

Dual Stream Independence Decoupling for True Emotion Recognition under Masked Expressions

Abstract

Recongnizing true emotions from masked expressions is extremely challenging due to deliberate concealment. Existing paradigms recognize true emotions from masked-expression clips that contain onsetframes just starting to disguise. However, this paradigm may not reflect the actual disguised state, as the onsetframe leaks the true emotional information without reaching a stable disguise state. Thus, this paper introduces a novel apexframe-based paradigm that classifies true emotions from the apexframe with a stable disguised state. Furthermore, this paper proposes a novel dual stream independence decoupling framework that decouples true and disguised emotion features, avoiding the interference of disguised emotions on true emotions. For efficient decoupling, we design a decoupling loss group, comprising two classification losses that learn true emotion and disguised expression features, respectively, and a Hilbert-Schmidt Independence loss that enhances the independence of two features. Experiments demonstrate that the apexframe-based paradigm is challenging, and the proposed decouple framework improves recogntion performances.
Paper Structure (22 sections, 4 equations, 2 figures, 5 tables)

This paper contains 22 sections, 4 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Recognition Paradigm
  4. Masked Expression Recognition
  5. Method
  6. Framework
  7. Backbone
  8. Masked Expression Adapter
  9. Dual Emotion Adapters
  10. Decoupling Loss Group
  11. Two Emotion Classification Losses ($\mathcal{L}_{\text{T}}$, $\mathcal{L}_{\text{D}}$)
  12. HSIC Feature Independence Loss ($\mathcal{L}_{\text{HSIC}}$)
  13. Experiments
  14. Dataset and Preprocessing
  15. Experimental Setup
  16. ...and 7 more sections

Figures (2)

  • Figure 1: The illustration of onset and apex frames from the masked expression sequence: a "happiness" disguised emotion with a "surprise" true emotion. The onset frame reveals the true emotion (surprise), while the apex frame reveals the disguised emotion (happiness).
  • Figure 2: Framework of dual stream independence decoupling model.