Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Culturally Adaptive Explainable LLM Assessment for Multilingual Information Disorder: A Human-in-the-Loop Approach

Maziar Kianimoghadam Jouneghani

Abstract

Recognizing information disorder is difficult because judgments about manipulation depend on cultural and linguistic context. Yet current Large Language Models (LLMs) often behave as monocultural, English-centric "black boxes," producing fluent rationales that overlook localized framing. Preliminary evidence from the multilingual Information Disorder (InDor) corpus suggests that existing models struggle to explain manipulated news consistently across communities. To address this gap, this ongoing study proposes a Hybrid Intelligence Loop, a human-in-the-loop (HITL) framework that grounds model assessment in human-written rationales from native-speaking annotators. The approach moves beyond static target-language few-shot prompting by pairing English task instructions with dynamically retrieved target-language exemplars drawn from filtered InDor annotations through In-Context Learning (ICL). In the initial pilot, the Exemplar Bank is seeded from these filtered annotations and used to compare static and adaptive prompting on Farsi and Italian news. The study evaluates span and severity prediction, the quality and cultural appropriateness of generated rationales, and model alignment across evaluator groups, providing a testbed for culturally grounded explainable AI.

Culturally Adaptive Explainable LLM Assessment for Multilingual Information Disorder: A Human-in-the-Loop Approach

Abstract

Recognizing information disorder is difficult because judgments about manipulation depend on cultural and linguistic context. Yet current Large Language Models (LLMs) often behave as monocultural, English-centric "black boxes," producing fluent rationales that overlook localized framing. Preliminary evidence from the multilingual Information Disorder (InDor) corpus suggests that existing models struggle to explain manipulated news consistently across communities. To address this gap, this ongoing study proposes a Hybrid Intelligence Loop, a human-in-the-loop (HITL) framework that grounds model assessment in human-written rationales from native-speaking annotators. The approach moves beyond static target-language few-shot prompting by pairing English task instructions with dynamically retrieved target-language exemplars drawn from filtered InDor annotations through In-Context Learning (ICL). In the initial pilot, the Exemplar Bank is seeded from these filtered annotations and used to compare static and adaptive prompting on Farsi and Italian news. The study evaluates span and severity prediction, the quality and cultural appropriateness of generated rationales, and model alignment across evaluator groups, providing a testbed for culturally grounded explainable AI.

Paper Structure

This paper contains 13 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Introduction and Motivation
  2. Starting Point: Baseline Evidence of Cross-Cultural Misalignment
  3. Proposed Methodology: The Hybrid Intelligence Loop
  4. Step 1: Problematic News Assessment
  5. Step 2: Expert Human Curation
  6. Step 3: Culturally Adaptive Prompting
  7. Pilot Evaluation Protocol
  8. Scope and Experimental Setup
  9. Operational Metrics
  10. Expected Findings and Limitations
  11. Hypothesized Outcomes
  12. Risks and Limitations
  13. Conclusion

Figures (3)

  • Figure 1: Structural transition from the original baseline to the proposed Culturally Adaptive prompt. The adaptive approach resolves the reasoning and cultural framing mismatch through retrieved target-culture exemplars.
  • Figure 2: The proposed Hybrid Intelligence Loop. The full architecture supports iterative expert feedback, while the initial pilot evaluates only the seeded Exemplar Bank phase before introducing continuous correction.
  • Figure 3: Visual breakdown of the four prompting conditions used in the pilot. M1 combines English task instructions with retrieved target-language cultural exemplars, while A1 keeps the same retrieval setup but switches the instruction language back to the target-language.