Acting Flatterers via LLMs Sycophancy: Combating Clickbait with LLMs Opposing-Stance Reasoning

TL;DR

This work reframes clickbait detection by exploiting LLM sycophancy to generate opposing-stance reasoning (agree/disagree) via the Self-renewal Opposing-Stance Reasoning Generation (SORG). The reasoning is then used in a local three-encoder BER(T) framework, ORCD, to perform contrastive learning with soft credibility labels and achieve robust binary classification. Across three benchmark datasets, ORCD outperforms prompting-based LLMs, small transformer models, and state-of-the-art baselines, while revealing the importance of each component through ablations and a systematic analysis of sycophancy levels. The approach offers a new interpretability-rich, contrastive signal pipeline for clickbait detection and suggests broader applicability to other discriminative NLP tasks.

Abstract

The widespread proliferation of online content has intensified concerns about clickbait, deceptive or exaggerated headlines designed to attract attention. While Large Language Models (LLMs) offer a promising avenue for addressing this issue, their effectiveness is often hindered by Sycophancy, a tendency to produce reasoning that matches users' beliefs over truthful ones, which deviates from instruction-following principles. Rather than treating sycophancy as a flaw to be eliminated, this work proposes a novel approach that initially harnesses this behavior to generate contrastive reasoning from opposing perspectives. Specifically, we design a Self-renewal Opposing-stance Reasoning Generation (SORG) framework that prompts LLMs to produce high-quality agree and disagree reasoning pairs for a given news title without requiring ground-truth labels. To utilize the generated reasoning, we develop a local Opposing Reasoning-based Clickbait Detection (ORCD) model that integrates three BERT encoders to represent the title and its associated reasoning. The model leverages contrastive learning, guided by soft labels derived from LLM-generated credibility scores, to enhance detection robustness. Experimental evaluations on three benchmark datasets demonstrate that our method consistently outperforms LLM prompting, fine-tuned smaller language models, and state-of-the-art clickbait detection baselines.

Figures (4)

• Figure 1: The figure shows the abstract workflow of our method. We leverage the capability of LLM Sycophancy to adjust two opposing stances in carefully designed prompts, making the LLM align with preset stance preferences to generate agree and disagree reasoning pairs, enabling the fine-tuned PLM to understand different interpretations of headlines and recognize potential clickbait elements.
• Figure 2: The system framework of ORCD: the opposing stance reasoning generated using our proposed SORG is fed into three Encoders to represent the title and the corresponding two opposing reasoning, respectively. The three presentations are then sent to two modules to conduct contrastive learning for the opposing stance reasoning - (1) the title-aware reasoning learner that embraces a cross-attention block and attention block to facilitate the interaction between titles and reasoning; (2) the title-free reasoning learner, which only contains an attention block to transform sequential representations to vectorial ones. It is noteworthy that the ratings of two reasoning generated $P_S = \{V_A, V_D\}$ by SORG are used as soft labels to train the two reasoning learners.
• Figure 3: Ablation comparisons on three datasets among the five variants of ORCD across three metrics. Specifically, w/o TF represents the original model that disables the title-free reasoning learner, w/o TA denotes that the title-aware reasoning learner is disabled in ORCD, w/o $\{V_A, V_D\}$ means soft labels is frozen to use, w/o $\{V_A, V_D\}$&TF implies that w/o TF and soft labels frozen are taken place simultaneously, vice versa for w/o $\{V_A, V_D\}$&TA.
• Figure 4: The alteration of the optimization objective $\hbar$ under the settings of different $\alpha$, $\beta$, and $\gamma$. The minimized $\hbar$ is marked in red color in the figure, which represents the best solution for SORG conduction.