Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Transformer and Prototype-based Interpretable Model for Contextual Sarcasm Detection

Ximing Wen, Rezvaneh Rezapour

TL;DR

This work tackles sarcasm detection by integrating semantic and sentiment embeddings from transformer models with a dual prototype-based architecture to enable intrinsic interpretability. The model uses semantic and sentiment prototype layers, initialized via class-specific clustering, and evaluates similarity with an RDF kernel, producing sentence-level explanations by projecting prototypes onto training samples. An incongruity loss between explicit and implicit sentiment pathways reinforces sarcasm signals, contributing to state-of-the-art performance on three public datasets. Empirical results, qualitative case studies, and an ablation analysis demonstrate that the approach delivers human-readable explanations while maintaining high predictive accuracy, highlighting its practical impact for transparent affective analysis. $sim(\mathbf{e}, \mathbf{p}) = \exp\left(-\frac{\|\mathbf{e}-\mathbf{p}\|_2^2}{\sigma^2}\right)$ is one example of the prototype similarity used to drive the interpretable decisions.

Abstract

Sarcasm detection, with its figurative nature, poses unique challenges for affective systems designed to perform sentiment analysis. While these systems typically perform well at identifying direct expressions of emotion, they struggle with sarcasm's inherent contradiction between literal and intended sentiment. Since transformer-based language models (LMs) are known for their efficient ability to capture contextual meanings, we propose a method that leverages LMs and prototype-based networks, enhanced by sentiment embeddings to conduct interpretable sarcasm detection. Our approach is intrinsically interpretable without extra post-hoc interpretability techniques. We test our model on three public benchmark datasets and show that our model outperforms the current state-of-the-art. At the same time, the prototypical layer enhances the model's inherent interpretability by generating explanations through similar examples in the reference time. Furthermore, we demonstrate the effectiveness of incongruity loss in the ablation study, which we construct using sentiment prototypes.

A Transformer and Prototype-based Interpretable Model for Contextual Sarcasm Detection

TL;DR

This work tackles sarcasm detection by integrating semantic and sentiment embeddings from transformer models with a dual prototype-based architecture to enable intrinsic interpretability. The model uses semantic and sentiment prototype layers, initialized via class-specific clustering, and evaluates similarity with an RDF kernel, producing sentence-level explanations by projecting prototypes onto training samples. An incongruity loss between explicit and implicit sentiment pathways reinforces sarcasm signals, contributing to state-of-the-art performance on three public datasets. Empirical results, qualitative case studies, and an ablation analysis demonstrate that the approach delivers human-readable explanations while maintaining high predictive accuracy, highlighting its practical impact for transparent affective analysis. is one example of the prototype similarity used to drive the interpretable decisions.

Abstract

Sarcasm detection, with its figurative nature, poses unique challenges for affective systems designed to perform sentiment analysis. While these systems typically perform well at identifying direct expressions of emotion, they struggle with sarcasm's inherent contradiction between literal and intended sentiment. Since transformer-based language models (LMs) are known for their efficient ability to capture contextual meanings, we propose a method that leverages LMs and prototype-based networks, enhanced by sentiment embeddings to conduct interpretable sarcasm detection. Our approach is intrinsically interpretable without extra post-hoc interpretability techniques. We test our model on three public benchmark datasets and show that our model outperforms the current state-of-the-art. At the same time, the prototypical layer enhances the model's inherent interpretability by generating explanations through similar examples in the reference time. Furthermore, we demonstrate the effectiveness of incongruity loss in the ablation study, which we construct using sentiment prototypes.

Paper Structure

This paper contains 35 sections, 12 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contextual Sarcasm Detection
  4. Explainability of Transformer Language Models
  5. Post hoc Interpretability:
  6. Prototype-based Reasoning in Deep Neural Networks:
  7. Methodology
  8. Semantic View
  9. Sentiment View
  10. Prototypical Layer
  11. Semantic Prototype:
  12. Sentiment Prototype Layer:
  13. Output Layer
  14. Loss and Training Algorithm
  15. Accuracy Loss:
  16. ...and 20 more sections

Figures (3)

  • Figure 1: Diagram of the proposed model architecture and workflow.
  • Figure 1: Case examples of input texts, semantic prototypes of SBERT and RoBERTa after projection, and the Euclidean distance score between the prototype embeddings and input sentence vector.
  • Figure 2: Visualization of SHAP values for words in the input text, with blue indicating negative contribution while red indicating positive. Darker color represents a larger contribution.