MRAC Track 1: 2nd Workshop on Multimodal, Generative and Responsible Affective Computing

TL;DR

The MRAC 2024 Track 1 workshop investigates responsible multimodal, generative affective computing, highlighting how Emotion AI relies on rich multimodal data and must address privacy, bias, and transparency while scaling from lab to real-world contexts. The contributions span four papers: (i) linear-attention multimodal transformers for video-based affect prediction with memory efficiency, (ii) expression-sensitive learning for macro- and micro-expression spotting across long videos, (iii) THE-FD, a hierarchical emotion-aware framework for precise fake-detection and localization using an emotional feature space, and (iv) W-TDL, a window-based temporal deepfake localization method combining audio-visual cues. Collectively, they advance robust, efficient, and ethically aware techniques for emotion understanding and deepfake detection in real-world multimedia. The work situates these methods within broader responsible AI considerations, including privacy preservation, bias mitigation, and explainability, aligned with ACM-MM themes and expert perspectives from leading researchers.

Abstract

With the rapid advancements in multimodal generative technology, Affective Computing research has provoked discussion about the potential consequences of AI systems equipped with emotional intelligence. Affective Computing involves the design, evaluation, and implementation of Emotion AI and related technologies aimed at improving people's lives. Designing a computational model in affective computing requires vast amounts of multimodal data, including RGB images, video, audio, text, and physiological signals. Moreover, Affective Computing research is deeply engaged with ethical considerations at various stages-from training emotionally intelligent models on large-scale human data to deploying these models in specific applications. Fundamentally, the development of any AI system must prioritize its impact on humans, aiming to augment and enhance human abilities rather than replace them, while drawing inspiration from human intelligence in a safe and responsible manner. The MRAC 2024 Track 1 workshop seeks to extend these principles from controlled, small-scale lab environments to real-world, large-scale contexts, emphasizing responsible development. The workshop also aims to highlight the potential implications of generative technology, along with the ethical consequences of its use, to researchers and industry professionals. To the best of our knowledge, this is the first workshop series to comprehensively address the full spectrum of multimodal, generative affective computing from a responsible AI perspective, and this is the second iteration of this workshop. Webpage: https://react-ws.github.io/2024/