Do Large Language Models Know Conflict? Investigating Parametric vs. Non-Parametric Knowledge of LLMs for Conflict Forecasting

TL;DR

This paper investigates whether large language models can forecast violent conflict from parametric knowledge alone and how retrieval-augmented generation (RAG) can supply up-to-date context. It compares GPT-4 and LLaMA-2 across 2020–2024 in the Horn of Africa and the Middle East, evaluating both zero-shot parametric forecasts and RAG-augmented predictions for conflict trends and fatalities. The results show that parametric knowledge captures broad patterns but struggles with fine-grained class distinctions and precise fatality estimates, while RAG consistently improves performance for GPT‑4 and, in some cases, enables competitive results for open models. The work provides a practical evaluation framework and suggests directions for multilingual retrieval, domain-specific fine-tuning, and human-in-the-loop systems to enable AI-assisted humanitarian forecasting.

Abstract

Large Language Models (LLMs) have shown impressive performance across natural language tasks, but their ability to forecast violent conflict remains underexplored. We investigate whether LLMs possess meaningful parametric knowledge-encoded in their pretrained weights-to predict conflict escalation and fatalities without external data. This is critical for early warning systems, humanitarian planning, and policy-making. We compare this parametric knowledge with non-parametric capabilities, where LLMs access structured and unstructured context from conflict datasets (e.g., ACLED, GDELT) and recent news reports via Retrieval-Augmented Generation (RAG). Incorporating external information could enhance model performance by providing up-to-date context otherwise missing from pretrained weights. Our two-part evaluation framework spans 2020-2024 across conflict-prone regions in the Horn of Africa and the Middle East. In the parametric setting, LLMs predict conflict trends and fatalities relying only on pretrained knowledge. In the non-parametric setting, models receive summaries of recent conflict events, indicators, and geopolitical developments. We compare predicted conflict trend labels (e.g., Escalate, Stable Conflict, De-escalate, Peace) and fatalities against historical data. Our findings highlight the strengths and limitations of LLMs for conflict forecasting and the benefits of augmenting them with structured external knowledge.