The Use of Artificial Intelligence in Military Intelligence: An Experimental Investigation of Added Value in the Analysis Process
Christian Nitzl, Achim Cyran, Sascha Krstanovic, Uwe M. Borghoff
TL;DR
The paper investigates the added value of artificial intelligence in military intelligence analysis using the deepCOM demonstrator, which combines AI search, automatic summarization, and Named Entity Recognition (NER) within a German-language interface. An experimental study with 50 source texts and a 30-minute analysis task compares AI-assisted analysis to a control condition using Bag-of-Words search, revealing that AI support improves overall performance on the first task part and enhances probability assessments, though self-reported confidence does not systematically increase. Post-hoc surveys indicate perceived speed gains and high usability (SUS ~86), with automated summarization seen as the most beneficial AI function, while NER and AI search receive more mixed ratings due to labeling errors and partial overlap with summarization. The study highlights practical benefits and limitations of AI in dynamic, ambiguous information environments, emphasizing the need for domain adaptation, explainable AI, and careful consideration of data quality and time constraints in real-world military analysis.
Abstract
It is beyond dispute that the potential benefits of artificial intelligence (AI) in military intelligence are considerable. Nevertheless, it remains uncertain precisely how AI can enhance the analysis of military data. The aim of this study is to address this issue. To this end, the AI demonstrator deepCOM was developed in collaboration with the start-up Aleph Alpha. The AI functions include text search, automatic text summarization and Named Entity Recognition (NER). These are evaluated for their added value in military analysis. It is demonstrated that under time pressure, the utilization of AI functions results in assessments clearly superior to that of the control group. Nevertheless, despite the demonstrably superior analysis outcome in the experimental group, no increase in confidence in the accuracy of their own analyses was observed. Finally, the paper identifies the limitations of employing AI in military intelligence, particularly in the context of analyzing ambiguous and contradictory information.