SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning

TL;DR

SarcasmMiner is proposed, a reinforcement learning based post-training framework that resists hallucination in multimodal reasoning and reasoning quality and suggests that reasoning-aware reward modeling enhances both performance and multimodal grounding.

Abstract

Multimodal sarcasm detection requires resolving pragmatic incongruity across textual, acoustic, and visual cues through cross-modal reasoning. To enable robust sarcasm reasoning with foundation models, we propose SarcasmMiner, a reinforcement learning based post-training framework that resists hallucination in multimodal reasoning. We reformulate sarcasm detection as structured reasoning and adopt a dual-track distillation strategy: high-quality teacher trajectories initialize the student model, while the full set of trajectories trains a generative reward model (GenRM) to evaluate reasoning quality. The student is optimized with group relative policy optimization (GRPO) using decoupled rewards for accuracy and reasoning quality. On MUStARD++, SarcasmMiner increases F1 from 59.83% (zero-shot), 68.23% (supervised finetuning) to 70.22%. These findings suggest that reasoning-aware reward modeling enhances both performance and multimodal grounding.

Figures (2)

• Figure 1: Overview of the proposed SarcasmMiner framework.
• Figure 2: Row-normalized confusion matrices for different post-training methods on the MUStARD++ dataset.