Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cognitive Insights Across Languages: Enhancing Multimodal Interview Analysis

David Ortiz-Perez, Jose Garcia-Rodriguez, David Tomás

TL;DR

The proposed model demonstrates the ability to transcribe and differentiate between languages used in the interviews, and extracts audio and text features, combining them into a multimodal architecture to achieve robust and generalized results.

Abstract

Cognitive decline is a natural process that occurs as individuals age. Early diagnosis of anomalous decline is crucial for initiating professional treatment that can enhance the quality of life of those affected. To address this issue, we propose a multimodal model capable of predicting Mild Cognitive Impairment and cognitive scores. The TAUKADIAL dataset is used to conduct the evaluation, which comprises audio recordings of clinical interviews. The proposed model demonstrates the ability to transcribe and differentiate between languages used in the interviews. Subsequently, the model extracts audio and text features, combining them into a multimodal architecture to achieve robust and generalized results. Our approach involves in-depth research to implement various features obtained from the proposed modalities.

Cognitive Insights Across Languages: Enhancing Multimodal Interview Analysis

TL;DR

The proposed model demonstrates the ability to transcribe and differentiate between languages used in the interviews, and extracts audio and text features, combining them into a multimodal architecture to achieve robust and generalized results.

Abstract

Cognitive decline is a natural process that occurs as individuals age. Early diagnosis of anomalous decline is crucial for initiating professional treatment that can enhance the quality of life of those affected. To address this issue, we propose a multimodal model capable of predicting Mild Cognitive Impairment and cognitive scores. The TAUKADIAL dataset is used to conduct the evaluation, which comprises audio recordings of clinical interviews. The proposed model demonstrates the ability to transcribe and differentiate between languages used in the interviews. Subsequently, the model extracts audio and text features, combining them into a multimodal architecture to achieve robust and generalized results. Our approach involves in-depth research to implement various features obtained from the proposed modalities.
Paper Structure (16 sections, 4 equations, 1 figure, 2 tables)

This paper contains 16 sections, 4 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Datasets
  4. Works
  5. Dataset
  6. Methodology
  7. Data preprocessing
  8. Text-based models
  9. Audio-based model
  10. Multimodal model
  11. Experiments
  12. Experimental setup
  13. Ablation study
  14. Challenge results
  15. Conclusions
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Overview of the multimodal architecture.