FABULA: Intelligence Report Generation Using Retrieval-Augmented Narrative Construction

TL;DR

FABULA tackles automatic intelligence report generation from noisy OSINT by uniting narrative theory with retrieval-augmented generation. It builds an Event Plot Graph (EPG) from OSINT using an Event Narrative Ontology (ENO) and extracts IPP based Lead, Body, and Tail plot points via NPCE. The pipeline fine-tunes GPT-Neo on the D+IR corpus and employs SPARQL-guided, prefix-tuned prompting to generate reports, aiming to reduce hallucinations and improve provenance. Quantitative ROUGE metrics and a qualitative human study demonstrate improvements in semantic relevance, narrative coherence, and data efficiency, suggesting practical benefits for intelligence analysis workflows.

Abstract

Narrative construction is the process of representing disparate event information into a logical plot structure that models an end to end story. Intelligence analysis is an example of a domain that can benefit tremendously from narrative construction techniques, particularly in aiding analysts during the largely manual and costly process of synthesizing event information into comprehensive intelligence reports. Manual intelligence report generation is often prone to challenges such as integrating dynamic event information, writing fine-grained queries, and closing information gaps. This motivates the development of a system that retrieves and represents critical aspects of events in a form that aids in automatic generation of intelligence reports. We introduce a Retrieval Augmented Generation (RAG) approach to augment prompting of an autoregressive decoder by retrieving structured information asserted in a knowledge graph to generate targeted information based on a narrative plot model. We apply our approach to the problem of neural intelligence report generation and introduce FABULA, framework to augment intelligence analysis workflows using RAG. An analyst can use FABULA to query an Event Plot Graph (EPG) to retrieve relevant event plot points, which can be used to augment prompting of a Large Language Model (LLM) during intelligence report generation. Our evaluation studies show that the plot points included in the generated intelligence reports have high semantic relevance, high coherency, and low data redundancy.

Figures (4)

• Figure 1: FABULA System Architecture and Data Flow.
• Figure 2: The Inverted Pyramid Plot (IPP) model and associated text features.
• Figure 3: Populated EPG Sub-graph for the 2023 Titan Submersible Implosion.
• Figure 4: FABULA's Retrieval-Augmented Generation (RAG) of Intelligence Report about event $e$.