A Large-Language-Model Framework for Automated Humanitarian Situation Reporting

TL;DR

This study demonstrates that generative AI can autonomously produce accurate, verifiable, and operationally useful humanitarian situation reports.

Abstract

Timely and accurate situational reports are essential for humanitarian decision-making, yet current workflows remain largely manual, resource intensive, and inconsistent. We present a fully automated framework that uses large language models (LLMs) to transform heterogeneous humanitarian documents into structured and evidence-grounded reports. The system integrates semantic text clustering, automatic question generation, retrieval augmented answer extraction with citations, multi-level summarization, and executive summary generation, supported by internal evaluation metrics that emulate expert reasoning. We evaluated the framework across 13 humanitarian events, including natural disasters and conflicts, using more than 1,100 documents from verified sources such as ReliefWeb. The generated questions achieved 84.7 percent relevance, 84.0 percent importance, and 76.4 percent urgency. The extracted answers reached 86.3 percent relevance, with citation precision and recall both exceeding 76 percent. Agreement between human and LLM based evaluations surpassed an F1 score of 0.80. Comparative analysis shows that the proposed framework produces reports that are more structured, interpretable, and actionable than existing baselines. By combining LLM reasoning with transparent citation linking and multi-level evaluation, this study demonstrates that generative AI can autonomously produce accurate, verifiable, and operationally useful humanitarian situation reports.

Figures (3)

• Figure 1: Overview of the proposed pipeline for transforming humanitarian documents into structured situational reports. The process comprises five main phases: semantic text clustering, automatic question generation, automatic answer extraction, creation of the summaries for each topic and for each SDG, executive summary generation,followed by a visualization module enabling four complementary modes of report exploration. Green boxes indicate components evaluated through automated metrics, while icons depicting human figures denote stages assessed by domain experts.
• Figure 2: This figure illustrates that our pipeline produces four different versions of the situational report, allowing the user to choose which one to view. The reports can be visualized either organized by Sustainable Development Goals (SDGs) or by automatically discovered sub-topics. Each version can be displayed in the form of a Q&A or as summarized text. All four versions include an executive summary at the beginning and provide citations for the specific claims made. Each citation includes the original paragraph, the document title, and a link to the source document.
• Figure 3: Expert evaluation of the three reporting systems: (i) our proposed framework, (ii) SmartBook, and (iii) a summarizer, across multiple dimensions of report quality. Evaluations were conducted by five humanitarian experts with direct operational experience in producing and interpreting situational reports. Responses are shown on a five-point Likert scale, ranging from "Strongly Disagree" (dark red) to "Strongly Agree" (dark green). For each criterion, coloured bars represent the number of expert ratings falling into each category. Higher concentrations of green indicate stronger perceived performance. The results show that our system is rated highest for analytical depth, integration of information, and support for humanitarian decision-making, while SmartBook performs strongly on ease of use. The summarizer receives predominantly neutral ratings due to its lack of structure and citations.