Nek Minit: Harnessing Pragmatic Metacognitive Prompting for Explainable Sarcasm Detection of Australian and Indian English

TL;DR

This work addresses explainable sarcasm detection across English varieties by leveraging Pragmatic Metacognitive Prompting (PMP) to inject regional and pragmatic context into generation. It extends the BESSTIE dataset with Australian and Indian English explanations (besstie-au, besstie-in) and evaluates PMP on two open-weight LLMs (GEMMA and LLAMA) against multiple baselines using the FLUTE dataset as standard American English reference. The results demonstrate statistically significant improvements (p ≤ 0.001) in both sarcasm detection accuracy and explanation quality across all datasets, with notable gains in cross-variety settings; agentic prompting (kg) further helps when external knowledge is required. The study provides a dataset, methodological framework, and insights into reasoning-aware prompting for explainable sarcasm, offering a pathway to more robust, region-aware NLP systems in sentiment analysis and related tasks.

Abstract

Sarcasm is a challenge to sentiment analysis because of the incongruity between stated and implied sentiment. The challenge is exacerbated when the implication may be relevant to a specific country or geographical region. Pragmatic metacognitive prompting (PMP) is a cognition-inspired technique that has been used for pragmatic reasoning. In this paper, we harness PMP for explainable sarcasm detection for Australian and Indian English, alongside a benchmark dataset for standard English. We manually add sarcasm explanations to an existing sarcasm-labeled dataset for Australian and Indian English called BESSTIE, and compare the performance for explainable sarcasm detection for them with FLUTE, a standard English dataset containing sarcasm explanations. Our approach utilising PMP when evaluated on two open-weight LLMs (GEMMA and LLAMA) achieves statistically significant performance improvement across all tasks and datasets when compared with four alternative prompting strategies. We also find that alternative techniques such as agentic prompting mitigate context-related failures by enabling external knowledge retrieval. The focused contribution of our work is utilising PMP in generating sarcasm explanations for varieties of English.

Figures (1)

• Figure 1: Evaluation methodology; (a) Dataset Creation in Section \ref{['sec:extend']}; (b) Prompting strategies in Section \ref{['sec:pmp-sec']}; (c) Evaluation tasks in Section \ref{['sec:method']}.