Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Curriculum Learning Meets Directed Acyclic Graph for Multimodal Emotion Recognition

Cam-Van Thi Nguyen, Cao-Bach Nguyen, Quang-Thuy Ha, Duc-Trong Le

TL;DR

A novel approach for Multimodal Emotion Recognition in Conversation that employs Directed Acyclic Graph (DAG) to integrate textual, acoustic, and visual features within a unified framework is proposed.

Abstract

Emotion recognition in conversation (ERC) is a crucial task in natural language processing and affective computing. This paper proposes MultiDAG+CL, a novel approach for Multimodal Emotion Recognition in Conversation (ERC) that employs Directed Acyclic Graph (DAG) to integrate textual, acoustic, and visual features within a unified framework. The model is enhanced by Curriculum Learning (CL) to address challenges related to emotional shifts and data imbalance. Curriculum learning facilitates the learning process by gradually presenting training samples in a meaningful order, thereby improving the model's performance in handling emotional variations and data imbalance. Experimental results on the IEMOCAP and MELD datasets demonstrate that the MultiDAG+CL models outperform baseline models. We release the code for MultiDAG+CL and experiments: https://github.com/vanntc711/MultiDAG-CL

Curriculum Learning Meets Directed Acyclic Graph for Multimodal Emotion Recognition

TL;DR

A novel approach for Multimodal Emotion Recognition in Conversation that employs Directed Acyclic Graph (DAG) to integrate textual, acoustic, and visual features within a unified framework is proposed.

Abstract

Emotion recognition in conversation (ERC) is a crucial task in natural language processing and affective computing. This paper proposes MultiDAG+CL, a novel approach for Multimodal Emotion Recognition in Conversation (ERC) that employs Directed Acyclic Graph (DAG) to integrate textual, acoustic, and visual features within a unified framework. The model is enhanced by Curriculum Learning (CL) to address challenges related to emotional shifts and data imbalance. Curriculum learning facilitates the learning process by gradually presenting training samples in a meaningful order, thereby improving the model's performance in handling emotional variations and data imbalance. Experimental results on the IEMOCAP and MELD datasets demonstrate that the MultiDAG+CL models outperform baseline models. We release the code for MultiDAG+CL and experiments: https://github.com/vanntc711/MultiDAG-CL
Paper Structure (17 sections, 8 equations, 2 figures, 4 tables, 1 algorithm)

This paper contains 17 sections, 8 equations, 2 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Multimodal ERC with Directed Acyclic Graph - MultiDAG
  4. Modality Encoder
  5. MultiDAG Construction
  6. Curriculum Learning - CL
  7. Difficulty Measure Function (DMF)
  8. Training Scheduler
  9. Experimental Setup
  10. Datasets and Baselines
  11. Implementation Details
  12. Results and Analysis
  13. Comparision with Baselines
  14. Effect of Modality
  15. Effect of Curriculum Learning
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overall structure of MultiDAG.
  • Figure 2: The confusion matrices on the IEMOCAP.