An independent evaluation of the Famine Early Warning Systems Network food security projections

TL;DR

The study provides an independent, geospatial evaluation of FEWSNET's near-term food security projections by aligning FEWSNET outputs with stable administrative boundaries and livelihood zones across 24 African countries and Yemen. It compares ML1 forecasts to the subsequent period's current assessments (CS) and benchmarks against rule-based baselines PPS, SPLY, and Max-2PP using a reproducible data-processing pipeline. The results show an overall accuracy of about 78%, with substantial regional and temporal variation and notable challenges in conflict-prone areas; FEWSNET generally outperforms simple heuristics, though performance declined slightly after 2020 and in certain countries like Somalia and South Sudan. The work advances a transparent, spatially aligned framework for forecast validation, informing methodological refinements and policy use in humanitarian planning.

Abstract

Reports from the Famine Early Warning Systems Network (FEWSNET) serve as the benchmark for food security predictions which is crucial for stakeholders in planning interventions and support people in need. This paper assesses the predictive accuracy of FEWSNET's food security forecasting, by comparing predictions to the following ground truth assessments at the administrative boundaries-livelihood level, revealing an overall high accuracy of 78\% across diverse timeframes and locations. However, our analysis also shows significant variations in performance across distinct regions and prediction periods. Therefore, our analysis sheds light on strengths, weaknesses, and areas for improvement in the context of food security predictions. The insights derived from this study not only enhance our understanding of FEWSNET's capabilities but also emphasize the importance of continuous refinement in forecasting methodologies.

Figures (8)

• Figure 1: Countries in blue are those for which FEWSNET generates regular assessments and projections.
• Figure 2: Source: fews.net
• Figure 3: Diagram of the data processing pipeline: in Step 1 we combine Administrative Boundaries and Livelihood Zones, in Step 2 we merge the output from the previous step with each published assessment and projections; Step 3 combines all the data into a single table, covering assessments and projections over time and geographies
• Figure 4: Example of visualisation of Administrative Boundaries and Livelihood zones in Somalia from FEWSNET shape files.
• Figure 5: Visualisation of the final output dataset, result of combining Administrative Boundaries, Livelihood Zones, and IPC FS assessment shape files published by FEWSNET (CS for February 2021)