Check News in One Click: NLP-Empowered Pro-Kremlin Propaganda Detection

TL;DR

The paper presents Check News in 1 Click, a multilingual NLP tool designed to detect pro-Kremlin propaganda across seven languages, delivering a verdict, probability, and explainable linguistic indicators. It combines language-specific BERT models and an SVM with linguistically derived features, exploring data augmentation via translation and per-language training to maximize detection performance. The dataset draws on Propaganda Diary DB and VoxCheck, with substantial augmentation, and the approach is evaluated through a user study that measures usefulness, learning, and perceived accuracy. Findings show high user satisfaction and the value of interpretable indicators, while language-specific biases and input-type variability highlight challenges in cross-language propaganda detection; the work demonstrates the practicality and user-centered design of an accessible anti-propaganda web interface with potential for broader media literacy applications.

Abstract

Many European citizens become targets of the Kremlin propaganda campaigns, aiming to minimise public support for Ukraine, foster a climate of mistrust and disunity, and shape elections (Meister, 2022). To address this challenge, we developed ''Check News in 1 Click'', the first NLP-empowered pro-Kremlin propaganda detection application available in 7 languages, which provides the lay user with feedback on their news, and explains manipulative linguistic features and keywords. We conducted a user study, analysed user entries and models' behaviour paired with questionnaire answers, and investigated the advantages and disadvantages of the proposed interpretative solution.

Figures (5)

• Figure 1: The figure illustrates the system's mock-up. The elliptical elements are rule-based reasoners while squared ones are trained models.
• Figure 2: The figure illustrates the distribution of the learnt features according to the stance. The upper red side shows the features with the highest negative coefficients for "Pro-Kremlin propaganda" prediction (hence, more likely in Western, Pro-Ukrainian media), while the lower blue side shows the coefficients indicative of "Pro-Kremlin propaganda".
• Figure 3: The figure illustrates statistics on the users who took part in the survey and used the application.
• Figure 4: The figure shows the results of the user study questionnaire.
• Figure 5: The figure illustrates differences in the text length between the training sub-corpora and the user inputs.