A Cross-Cultural Assessment of Human Ability to Detect LLM-Generated Fake News about South Africa
Tim Schlippe, Matthias Wölfel, Koena Ronny Mabokela
TL;DR
This study investigates how cultural proximity influences the ability to detect AI-generated fake news about South Africa by comparing South African and international readers. An online survey (n=89) presents 20 articles (10 true, 10 AI-generated via GPT-4o with indirect prompting), measuring detection accuracy as deviations from an ideal rating and analyzing cue usage. Results show a cross-cultural split: South Africans excel at recognizing true South African news but struggle more with fake content, while participants from other nationalities perform better on fake content, with similar overall detection difficulty across groups. Objective linguistic analyses reveal differences in readability and sentiment between true and fake articles, but these signals do not consistently drive human judgments, highlighting the need for hybrid detection strategies and culturally informed interventions.
Abstract
This study investigates how cultural proximity affects the ability to detect AI-generated fake news by comparing South African participants with those from other nationalities. As large language models increasingly enable the creation of sophisticated fake news, understanding human detection capabilities becomes crucial, particularly across different cultural contexts. We conducted a survey where 89 participants (56 South Africans, 33 from other nationalities) evaluated 10 true South African news articles and 10 AI-generated fake versions. Results reveal an asymmetric pattern: South Africans demonstrated superior performance in detecting true news about their country (40% deviation from ideal rating) compared to other participants (52%), but performed worse at identifying fake news (62% vs. 55%). This difference may reflect South Africans' higher overall trust in news sources. Our analysis further shows that South Africans relied more on content knowledge and contextual understanding when judging credibility, while participants from other countries emphasised formal linguistic features such as grammar and structure. Overall, the deviation from ideal rating was similar between groups (51% vs. 53%), suggesting that cultural familiarity appears to aid verification of authentic information but may also introduce bias when evaluating fabricated content. These insights contribute to understanding cross-cultural dimensions of misinformation detection and inform strategies for combating AI-generated fake news in increasingly globalised information ecosystems where content crosses cultural and geographical boundaries.