This Paper Had the Smartest Reviewers -- Flattery Detection Utilising an Audio-Textual Transformer-Based Approach

TL;DR

This work tackles automatic detection of flattery in spoken language by introducing a novel 20-hour audio-textual dataset derived from earnings-call style interactions and by training text-only, audio-only, and multimodal classifiers. Text-based RoBERTa models on gold transcripts achieve around 85.97% Unweighted Average Recall (UAR) on the test set, while audio models based on AST, Wav2Vec2, and Whisper reach approximately 82% UAR; combining modalities yields the strongest performance with up to 87.16% UAR. The study demonstrates that textual information is the primary signal for flattery detection, but audio cues provide complementary information that improves detection, particularly when transcripts are noisy, and confirms that early fusion of modalities generally outperforms late fusion. Limitations include demographic biases in the dataset (underrepresentation of female speakers) and domain-specific language, motivating future work to broaden demographics, incorporate larger contextual units, and develop more advanced fusion strategies.

Abstract

Flattery is an important aspect of human communication that facilitates social bonding, shapes perceptions, and influences behavior through strategic compliments and praise, leveraging the power of speech to build rapport effectively. Its automatic detection can thus enhance the naturalness of human-AI interactions. To meet this need, we present a novel audio textual dataset comprising 20 hours of speech and train machine learning models for automatic flattery detection. In particular, we employ pretrained AST, Wav2Vec2, and Whisper models for the speech modality, and Whisper TTS models combined with a RoBERTa text classifier for the textual modality. Subsequently, we build a multimodal classifier by combining text and audio representations. Evaluation on unseen test data demonstrates promising results, with Unweighted Average Recall scores reaching 82.46% in audio-only experiments, 85.97% in text-only experiments, and 87.16% using a multimodal approach.