Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing Multimodal Teacher Sentiment Analysis:The Large-Scale T-MED Dataset & The Effective AAM-TSA Model

Zhiyi Duan, Xiangren Wang, Hongyu Yuan, Qianli Xing

TL;DR

The paper tackles the challenge of interpreting teacher emotions in real classrooms by leveraging multimodal signals and instructional context. It introduces T-MED, a large-scale dataset with 14,938 samples across text, audio, video, and instructional information, annotated via a human-machine collaboration, and AAM-TSA, an asymmetric attention-based multimodal model that fuses modalities with an audio-centric cross-modal mechanism. Empirical results on T-MED show that AAM-TSA outperforms nine baselines in WA and W-F1, with notable gains in fine-grained emotion recognition and interpretability. The work advances educational affective computing and enables more accurate, context-aware sentiment analysis for teacher analytics and educational interventions.

Abstract

Teachers' emotional states are critical in educational scenarios, profoundly impacting teaching efficacy, student engagement, and learning achievements. However, existing studies often fail to accurately capture teachers' emotions due to the performative nature and overlook the critical impact of instructional information on emotional expression.In this paper, we systematically investigate teacher sentiment analysis by building both the dataset and the model accordingly. We construct the first large-scale teacher multimodal sentiment analysis dataset, T-MED.To ensure labeling accuracy and efficiency, we employ a human-machine collaborative labeling process.The T-MED dataset includes 14,938 instances of teacher emotional data from 250 real classrooms across 11 subjects ranging from K-12 to higher education, integrating multimodal text, audio, video, and instructional information.Furthermore, we propose a novel asymmetric attention-based multimodal teacher sentiment analysis model, AAM-TSA.AAM-TSA introduces an asymmetric attention mechanism and hierarchical gating unit to enable differentiated cross-modal feature fusion and precise emotional classification. Experimental results demonstrate that AAM-TSA significantly outperforms existing state-of-the-art methods in terms of accuracy and interpretability on the T-MED dataset.

Advancing Multimodal Teacher Sentiment Analysis:The Large-Scale T-MED Dataset & The Effective AAM-TSA Model

TL;DR

The paper tackles the challenge of interpreting teacher emotions in real classrooms by leveraging multimodal signals and instructional context. It introduces T-MED, a large-scale dataset with 14,938 samples across text, audio, video, and instructional information, annotated via a human-machine collaboration, and AAM-TSA, an asymmetric attention-based multimodal model that fuses modalities with an audio-centric cross-modal mechanism. Empirical results on T-MED show that AAM-TSA outperforms nine baselines in WA and W-F1, with notable gains in fine-grained emotion recognition and interpretability. The work advances educational affective computing and enables more accurate, context-aware sentiment analysis for teacher analytics and educational interventions.

Abstract

Teachers' emotional states are critical in educational scenarios, profoundly impacting teaching efficacy, student engagement, and learning achievements. However, existing studies often fail to accurately capture teachers' emotions due to the performative nature and overlook the critical impact of instructional information on emotional expression.In this paper, we systematically investigate teacher sentiment analysis by building both the dataset and the model accordingly. We construct the first large-scale teacher multimodal sentiment analysis dataset, T-MED.To ensure labeling accuracy and efficiency, we employ a human-machine collaborative labeling process.The T-MED dataset includes 14,938 instances of teacher emotional data from 250 real classrooms across 11 subjects ranging from K-12 to higher education, integrating multimodal text, audio, video, and instructional information.Furthermore, we propose a novel asymmetric attention-based multimodal teacher sentiment analysis model, AAM-TSA.AAM-TSA introduces an asymmetric attention mechanism and hierarchical gating unit to enable differentiated cross-modal feature fusion and precise emotional classification. Experimental results demonstrate that AAM-TSA significantly outperforms existing state-of-the-art methods in terms of accuracy and interpretability on the T-MED dataset.
Paper Structure (31 sections, 24 equations, 3 figures, 6 tables)

This paper contains 31 sections, 24 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Relate Work
  3. Teachers' Sentiment Analysis
  4. Datasets for Teachers' Sentiment Analysis
  5. Human-Machine Collaboration Annotation Framework for T-MED Dataset
  6. Asymmetric Attention-based Multimodal Teacher Sentiment Analysis Model
  7. Multimodal Feature Extraction Module
  8. Asymmetric Cross-Modal Interaction Module
  9. Hierarchical Feature Fusion Module
  10. Emotion Classification Module
  11. Experiment
  12. Experimental Setup
  13. Performance Comparison on T-MED
  14. Fine-grained Sentiment Analysis
  15. Case Study: Teacher Sentiment Analysis in a Biology Classroom From High School
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Diagram of T-MED dataset construction process.
  • Figure 2: AAM-TSA model architecture diagram.
  • Figure 3: Analysis of predictive case.